¿Cómo puedo normalizar el carácter EOL en Java?

Question

Tengo un servidor Linux y muchos clientes con muchos sistemas operativos. El servidor toma un archivo de entrada de los clientes. Linux tiene fin de línea char LF, mientras que Mac tiene fin de línea char CR, y Windows tiene fin de línea char CR + LF

El servidor necesita como fin de línea char LF. Usando java, quiero asegurarme de que el archivo siempre use el linux eol char LF. ¿Cómo puedo lograrlo?

Answer 1

La combinación de las dos respuestas (por Visage & eumiro):

EDIT:Después de leer el comentario. línea. System.getProperty ("line.separator") no tiene uso entonces.
Antes de enviar el archivo al servidor, abra que reemplazar todos los EOLs y reescritura
Asegúrese de utilizar flujos de datos para hacerlo, y escribir en binario

String fileString; 
//.. 
//read from the file 
//.. 
//for windows 
fileString = fileString.replaceAll("\\r\\n", "\n"); 
fileString = fileString.replaceAll("\\r", "\n"); 
//.. 
//write to file in binary mode.. something like: 
DataOutputStream os = new DataOutputStream(new FileOutputStream("fname.txt")); 
os.write(fileString.getBytes()); 
//.. 
//send file 
//.. 

el método replaceAll tiene dos argumentos, el primero uno es la cadena para reemplazar y el segundo es el reemplazo. Pero, el primero se trata como una expresión regular, por lo tanto, '\' se interpreta de esa manera. Entonces:

"\\r\\n" is converted to "\r\n" by Regex 
"\r\n" is converted to CR+LR by java 

Answer 2

¿Podría probar esto?

content.replaceAll("\\r\\n?", "\n") 

Answer 3

Uso

System.getProperty("line.separator") 

que le dará el carácter (s) (local) EOL. A continuación, puede utilizar un análisis del incomible para determinar qué 'sabor' es y convertir en consecuencia.

Alternativamente, conseguir a sus clientes a estandarizar!

Answer 4

Tuve que hacer esto para un proyecto reciente. El siguiente método normalizará las terminaciones de línea en el archivo dado hasta el final de línea especificado por el sistema operativo en el que se ejecuta la JVM. Entonces, si JVM se ejecuta en Linux, esto normalizará todas las terminaciones de línea a LF (\ n).

También funciona en archivos muy grandes debido al uso de flujos de búfer.

public static void normalizeFile(File f) {  
    File temp = null; 
    BufferedReader bufferIn = null; 
    BufferedWriter bufferOut = null;   

    try {   
     if(f.exists()) { 
      // Create a new temp file to write to 
      temp = new File(f.getAbsolutePath() + ".normalized"); 
      temp.createNewFile(); 

      // Get a stream to read from the file un-normalized file 
      FileInputStream fileIn = new FileInputStream(f); 
      DataInputStream dataIn = new DataInputStream(fileIn); 
      bufferIn = new BufferedReader(new InputStreamReader(dataIn)); 

      // Get a stream to write to the normalized file 
      FileOutputStream fileOut = new FileOutputStream(temp); 
      DataOutputStream dataOut = new DataOutputStream(fileOut); 
      bufferOut = new BufferedWriter(new OutputStreamWriter(dataOut)); 

      // For each line in the un-normalized file 
      String line; 
      while ((line = bufferIn.readLine()) != null) { 
       // Write the original line plus the operating-system dependent newline 
       bufferOut.write(line); 
       bufferOut.newLine();         
      } 

      bufferIn.close(); 
      bufferOut.close(); 

      // Remove the original file 
      f.delete(); 

      // And rename the original file to the new one 
      temp.renameTo(f); 
     } else { 
      // If the file doesn't exist... 
      log.warn("Could not find file to open: " + f.getAbsolutePath()); 
     } 
    } catch (Exception e) { 
     log.warn(e.getMessage(), e); 
    } finally { 
     // Clean up, temp should never exist 
     FileUtils.deleteQuietly(temp); 
     IOUtils.closeQuietly(bufferIn); 
     IOUtils.closeQuietly(bufferOut); 
    } 
} 

Answer 5

Aquí hay una clase completa de ayuda para tratar los problemas de EOL. Está parcialmente basado en la solución publicada por tyjen.

import java.io.BufferedInputStream; 
import java.io.BufferedReader; 
import java.io.BufferedWriter; 
import java.io.DataInputStream; 
import java.io.DataOutputStream; 
import java.io.File; 
import java.io.FileInputStream; 
import java.io.FileOutputStream; 
import java.io.IOException; 
import java.io.InputStreamReader; 
import java.io.OutputStreamWriter; 

import org.apache.commons.io.FileUtils; 
import org.apache.commons.io.IOUtils; 

/** 
* Helper class to deal with end-of-line markers in text files. 
* 
* Loosely based on these examples: 
* - http://stackoverflow.com/a/9456947/1084488 (cc by-sa 3.0) 
* - http://svn.apache.org/repos/asf/tomcat/trunk/java/org/apache/tomcat/buildutil/CheckEol.java (Apache License v2.0) 
* 
* This file is posted here to meet the "ShareAlike" requirement of cc by-sa 3.0: 
* http://stackoverflow.com/a/27930311/1084488 
* 
* @author Matthias Stevens 
*/ 
public class EOLUtils 
{ 

    /** 
    * Unix-style end-of-line marker (LF) 
    */ 
    private static final String EOL_UNIX = "\n"; 

    /** 
    * Windows-style end-of-line marker (CRLF) 
    */ 
    private static final String EOL_WINDOWS = "\r\n"; 

    /** 
    * "Old Mac"-style end-of-line marker (CR) 
    */ 
    private static final String EOL_OLD_MAC = "\r"; 

    /** 
    * Default end-of-line marker on current system 
    */ 
    private static final String EOL_SYSTEM_DEFAULT = System.getProperty("line.separator"); 

    /** 
    * The support end-of-line marker modes 
    */ 
    public static enum Mode 
    { 
     /** 
     * Unix-style end-of-line marker ("\n") 
     */ 
     LF, 

     /** 
     * Windows-style end-of-line marker ("\r\n") 
     */ 
     CRLF, 

     /** 
     * "Old Mac"-style end-of-line marker ("\r") 
     */ 
     CR 
    } 

    /** 
    * The default end-of-line marker mode for the current system 
    */ 
    public static final Mode SYSTEM_DEFAULT = (EOL_SYSTEM_DEFAULT.equals(EOL_UNIX) ? Mode.LF : (EOL_SYSTEM_DEFAULT 
     .equals(EOL_WINDOWS) ? Mode.CRLF : (EOL_SYSTEM_DEFAULT.equals(EOL_OLD_MAC) ? Mode.CR : null))); 
    static 
    { 
     // Just in case... 
     if (SYSTEM_DEFAULT == null) 
     { 
      throw new IllegalStateException("Could not determine system default end-of-line marker"); 
     } 
    } 

    /** 
    * Determines the end-of-line {@link Mode} of a text file. 
    * 
    * @param textFile the file to investigate 
    * @return the end-of-line {@link Mode} of the given file, or {@code null} if it could not be determined 
    * @throws Exception 
    */ 
    public static Mode determineEOL(File textFile) 
     throws Exception 
    { 
     if (!textFile.exists()) 
     { 
      throw new IOException("Could not find file to open: " + textFile.getAbsolutePath()); 
     } 

     FileInputStream fileIn = new FileInputStream(textFile); 
     BufferedInputStream bufferIn = new BufferedInputStream(fileIn); 
     try 
     { 
      int prev = -1; 
      int ch; 
      while ((ch = bufferIn.read()) != -1) 
      { 
       if (ch == '\n') 
       { 
        if (prev == '\r') 
        { 
         return Mode.CRLF; 
        } 
        else 
        { 
         return Mode.LF; 
        } 
       } 
       else if (prev == '\r') 
       { 
        return Mode.CR; 
       } 
       prev = ch; 
      } 
      throw new Exception("Could not determine end-of-line marker mode"); 
     } 
     catch (IOException ioe) 
     { 
      throw new Exception("Could not determine end-of-line marker mode", ioe); 
     } 
     finally 
     { 
      // Clean up: 
      IOUtils.closeQuietly(bufferIn); 
     } 
    } 

    /** 
    * Checks whether the given text file has Windows-style (CRLF) line endings. 
    * 
    * @param textFile the file to investigate 
    * @return 
    * @throws Exception 
    */ 
    public static boolean hasWindowsEOL(File textFile) 
     throws Exception 
    { 
     return Mode.CRLF.equals(determineEOL(textFile)); 
    } 

    /** 
    * Checks whether the given text file has Unix-style (LF) line endings. 
    * 
    * @param textFile the file to investigate 
    * @return 
    * @throws Exception 
    */ 
    public static boolean hasUnixEOL(File textFile) 
     throws Exception 
    { 
     return Mode.LF.equals(determineEOL(textFile)); 
    } 

    /** 
    * Checks whether the given text file has "Old Mac"-style (CR) line endings. 
    * 
    * @param textFile the file to investigate 
    * @return 
    * @throws Exception 
    */ 
    public static boolean hasOldMacEOL(File textFile) 
     throws Exception 
    { 
     return Mode.CR.equals(determineEOL(textFile)); 
    } 

    /** 
    * Checks whether the given text file has line endings that conform to the system default mode (e.g. LF on Unix). 
    * 
    * @param textFile the file to investigate 
    * @return 
    * @throws Exception 
    */ 
    public static boolean hasSystemDefaultEOL(File textFile) 
     throws Exception 
    { 
     return SYSTEM_DEFAULT.equals(determineEOL(textFile)); 
    } 

    /** 
    * Convert the line endings in the given file to Unix-style (LF). 
    * 
    * @param textFile the file to process 
    * @throws IOException 
    */ 
    public static void convertToUnixEOL(File textFile) 
     throws IOException 
    { 
     convertLineEndings(textFile, EOL_UNIX); 
    } 

    /** 
    * Convert the line endings in the given file to Windows-style (CRLF). 
    * 
    * @param textFile the file to process 
    * @throws IOException 
    */ 
    public static void convertToWindowsEOL(File textFile) 
     throws IOException 
    { 
     convertLineEndings(textFile, EOL_WINDOWS); 
    } 

    /** 
    * Convert the line endings in the given file to "Old Mac"-style (CR). 
    * 
    * @param textFile the file to process 
    * @throws IOException 
    */ 
    public static void convertToOldMacEOL(File textFile) 
     throws IOException 
    { 
     convertLineEndings(textFile, EOL_OLD_MAC); 
    } 

    /** 
    * Convert the line endings in the given file to the system default mode. 
    * 
    * @param textFile the file to process 
    * @throws IOException 
    */ 
    public static void convertToSystemEOL(File textFile) 
     throws IOException 
    { 
     convertLineEndings(textFile, EOL_SYSTEM_DEFAULT); 
    } 

    /** 
    * Line endings conversion method. 
    * 
    * @param textFile the file to process 
    * @param eol the end-of-line marker to use (as a {@link String}) 
    * @throws IOException 
    */ 
    private static void convertLineEndings(File textFile, String eol) 
     throws IOException 
    { 
     File temp = null; 
     BufferedReader bufferIn = null; 
     BufferedWriter bufferOut = null; 

     try 
     { 
      if (textFile.exists()) 
      { 
       // Create a new temp file to write to 
       temp = new File(textFile.getAbsolutePath() + ".normalized"); 
       temp.createNewFile(); 

       // Get a stream to read from the file un-normalized file 
       FileInputStream fileIn = new FileInputStream(textFile); 
       DataInputStream dataIn = new DataInputStream(fileIn); 
       bufferIn = new BufferedReader(new InputStreamReader(dataIn)); 

       // Get a stream to write to the normalized file 
       FileOutputStream fileOut = new FileOutputStream(temp); 
       DataOutputStream dataOut = new DataOutputStream(fileOut); 
       bufferOut = new BufferedWriter(new OutputStreamWriter(dataOut)); 

       // For each line in the un-normalized file 
       String line; 
       while ((line = bufferIn.readLine()) != null) 
       { 
        // Write the original line plus the operating-system dependent newline 
        bufferOut.write(line); 
        bufferOut.write(eol); // write EOL marker 
       } 

       // Close buffered reader & writer: 
       bufferIn.close(); 
       bufferOut.close(); 

       // Remove the original file 
       textFile.delete(); 

       // And rename the original file to the new one 
       temp.renameTo(textFile); 
      } 
      else 
      { 
       // If the file doesn't exist... 
       throw new IOException("Could not find file to open: " + textFile.getAbsolutePath()); 
      } 
     } 
     finally 
     { 
      // Clean up, temp should never exist 
      FileUtils.deleteQuietly(temp); 
      IOUtils.closeQuietly(bufferIn); 
      IOUtils.closeQuietly(bufferOut); 
     } 
    } 

} 

Answer 6

public static String normalize(String val) { 
    return val.replace("\r\n", "\n") 
      .replace("\r", "\n"); 
} 

Para HTML:

public static String normalize(String val) { 
    return val.replace("\r\n", "<br/>") 
      .replace("\n", "<br/>") 
      .replace("\r", "<br/>"); 
} 

Answer 7

solución para cambiar el archivo que termina con la búsqueda recursiva en el camino

package handleFileLineEnd; 

import java.io.File; 
import java.io.IOException; 
import java.nio.charset.Charset; 
import java.nio.charset.StandardCharsets; 
import java.nio.file.FileSystems; 
import java.nio.file.Files; 
import java.nio.file.OpenOption; 
import java.nio.file.Path; 
import java.nio.file.Paths; 
import java.util.List; 

import sun.awt.image.BytePackedRaster; 

public class handleFileEndingMain { 

    static int carriageReturnTotal; 
    static int newLineTotal; 

    public static void main(String[] args) throws IOException 
    {  
     processPath("c:/temp/directories"); 

     System.out.println("carriageReturnTotal (files have issue): " + carriageReturnTotal); 

     System.out.println("newLineTotal: " + newLineTotal); 
    } 

    private static void processPath(String path) throws IOException 
    { 
     File dir = new File(path); 
     File[] directoryListing = dir.listFiles(); 

     if (directoryListing != null) { 
      for (File child : directoryListing) { 
       if (child.isDirectory())     
        processPath(child.toString());    
       else 
        checkFile(child.toString()); 
      } 
     } 


    } 

    private static void checkFile(String fileName) throws IOException 
    { 
     Path path = FileSystems.getDefault().getPath(fileName); 

     byte[] bytes= Files.readAllBytes(path); 

     for (int counter=0; counter<bytes.length; counter++) 
     { 
      if (bytes[counter] == 13) 
      { 
       carriageReturnTotal = carriageReturnTotal + 1; 

       System.out.println(fileName); 
       modifyFile(fileName); 
       break; 
      } 
      if (bytes[counter] == 10) 
      { 
       newLineTotal = newLineTotal+ 1; 
       //System.out.println(fileName); 
       break; 
      } 
     } 

    } 

    private static void modifyFile(String fileName) throws IOException 
    { 

     Path path = Paths.get(fileName); 
     Charset charset = StandardCharsets.UTF_8; 

     String content = new String(Files.readAllBytes(path), charset); 
     content = content.replaceAll("\r\n", "\n"); 
     content = content.replaceAll("\r", "\n"); 
     Files.write(path, content.getBytes(charset)); 
    } 
} 

Answer 8

Aunque String.replaceAll() es más fácil de código, este debería funcionar mejor ya que no pasa por la infraestructura de expresiones regulares.

/** 
* Accepts a non-null string and returns the string with all end-of-lines 
* normalized to a \n. This means \r\n and \r will both be normalized to \n. 
* <p> 
*  Impl Notes: Although regex would have been easier to code, this approach 
*  will be more efficient since it's purpose built for this use case. Note we only 
*  construct a new StringBuilder and start appending to it if there are new end-of-lines 
*  to be normalized found in the string. If there are no end-of-lines to be replaced 
*  found in the string, this will simply return the input value. 
* </p> 
* 
* @param inputValue !null, input value that may or may not contain new lines 
* @return the input value that has new lines normalized 
*/ 
static String normalizeNewLines(String inputValue){ 
    StringBuilder stringBuilder = null; 
    int index = 0; 
    int len = inputValue.length(); 
    while (index < len){ 
     char c = inputValue.charAt(index); 
     if (c == '\r'){ 
      if (stringBuilder == null){ 
       stringBuilder = new StringBuilder(); 
       // build up the string builder so it contains all the prior characters 
       stringBuilder.append(inputValue.substring(0, index)); 
      } 
      if ((index + 1 < len) && 
       inputValue.charAt(index + 1) == '\n'){ 
       // this means we encountered a \r\n ... move index forward one more character 
       index++; 
      } 
      stringBuilder.append('\n'); 
     }else{ 
      if (stringBuilder != null){ 
       stringBuilder.append(c); 
      } 
     } 
     index++; 
    } 
    return stringBuilder == null ? inputValue : stringBuilder.toString(); 
}