¿Cómo puedo dividir un archivo en python?

Question

¿Es posible dividir un archivo? Por ejemplo, tiene una enorme lista de palabras, quiero dividirla para que se convierta en más de un archivo. ¿Cómo es esto posible?

Answer 1

Claro que es posible:

open input file 
open output file 1 
count = 0 
for each line in file: 
    write to output file 
    count = count + 1 
    if count > maxlines: 
     close output file 
     open next output file 
     count = 0 

Answer 2

Ésta divide un archivo por saltos de línea y lo escribe de vuelta. Puedes cambiar el delimitador fácilmente. Esto también puede manejar cantidades desiguales, si no tiene un múltiplo de líneas splitLen (20 en este ejemplo) en su archivo de entrada.

splitLen = 20   # 20 lines per file 
outputBase = 'output' # output.1.txt, output.2.txt, etc. 

# This is shorthand and not friendly with memory 
# on very large files (Sean Cavanagh), but it works. 
input = open('input.txt', 'r').read().split('\n') 

at = 1 
for lines in range(0, len(input), splitLen): 
    # First, get the list slice 
    outputData = input[lines:lines+splitLen] 

    # Now open the output file, join the new slice with newlines 
    # and write it out. Then close the file. 
    output = open(outputBase + str(at) + '.txt', 'w') 
    output.write('\n'.join(outputData)) 
    output.close() 

    # Increment the counter 
    at += 1 

Answer 3

Solución para dividir archivos binarios en capítulos .000, .001, etc .: memoria

FILE = 'scons-conversion.7z' 

MAX = 500*1024*1024 # 500Mb - max chapter size 
BUF = 50*1024*1024*1024 # 50GB - memory buffer size 

chapters = 0 
uglybuf = '' 
with open(FILE, 'rb') as src: 
    while True: 
    tgt = open(FILE + '.%03d' % chapters, 'wb') 
    written = 0 
    while written < MAX: 
     if len(uglybuf) > 0: 
     tgt.write(uglybuf) 
     tgt.write(src.read(min(BUF, MAX - written))) 
     written += min(BUF, MAX - written) 
     uglybuf = src.read(1) 
     if len(uglybuf) == 0: 
     break 
    tgt.close() 
    if len(uglybuf) == 0: 
     break 
    chapters += 1 

Answer 4

Una mejor bucle, por ejemplo, de SLI, no acaparando:

splitLen = 20   # 20 lines per file 
outputBase = 'output' # output.1.txt, output.2.txt, etc. 

input = open('input.txt', 'r') 

count = 0 
at = 0 
dest = None 
for line in input: 
    if count % splitLen == 0: 
     if dest: dest.close() 
     dest = open(outputBase + str(at) + '.txt', 'w') 
     at += 1 
    dest.write(line) 
    count += 1 

Answer 5

def split_file(file, prefix, max_size, buffer=1024): 
    """ 
    file: the input file 
    prefix: prefix of the output files that will be created 
    max_size: maximum size of each created file in bytes 
    buffer: buffer size in bytes 

    Returns the number of parts created. 
    """ 
    with open(file, 'r+b') as src: 
     suffix = 0 
     while True: 
      with open(prefix + '.%s' % suffix, 'w+b') as tgt: 
       written = 0 
       while written < max_size: 
        data = src.read(buffer) 
        if data: 
         tgt.write(data) 
         written += buffer 
        else: 
         return suffix 
       suffix += 1 


def cat_files(infiles, outfile, buffer=1024): 
    """ 
    infiles: a list of files 
    outfile: the file that will be created 
    buffer: buffer size in bytes 
    """ 
    with open(outfile, 'w+b') as tgt: 
     for infile in sorted(infiles): 
      with open(infile, 'r+b') as src: 
       while True: 
        data = src.read(buffer) 
        if data: 
         tgt.write(data) 
        else: 
         break 

Answer 6

import re 
PATENTS = 'patent.data' 

def split_file(filename): 
    # Open file to read 
    with open(filename, "r") as r: 

     # Counter 
     n=0 

     # Start reading file line by line 
     for i, line in enumerate(r): 

      # If line match with teplate -- <?xml --increase counter n 
      if re.match(r'\<\?xml', line): 
       n+=1 

       # This "if" can be deleted, without it will start naming from 1 
       # or you can keep it. It depends where is "re" will find at 
       # first time the template. In my case it was first line 
       if i == 0: 
        n = 0    

      # Write lines to file  
      with open("{}-{}".format(PATENTS, n), "a") as f: 
       f.write(line) 

split_file(PATENTS) 

Como resultado obtendrá:

patent.data-0

patent.data-1

patent.data-N

Answer 7

Puedes utilizar este módulo PyPI filesplit.