> I can sort the biggest logfile (800M) using unix sort in about 1.5
> mins on my workstation. That's not really fast enough, with
> potentially 12 other files....
You won't beat sort with Python.
You have to be realistic, these are very big files!
Python should be faster overall but for specific tasks the Unix
tools written in C will be faster.
But if you are merging multiple files into one then sorting
them before processing will probably help. However if you expect
to be pruning out more lines than you keep it might be easier just
to throw all the data you want into a single file and then sort that
at the end. It all depends on the data.
HTH,
Alan G
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
