Thanks Shaggs, but I work on Linux and Mac OS/X. My script will combine multiple overlapping datasets for a single security into one sorted non-overlapping set. Emacs (or vi, or ...) will "handle" the memory requirements to load files this size also, my problem was combining files and removing duplicate lines. Otherwise I'd have used the "cat" command to concatenate all the files into one. :)
Why do I have overlapping datasets? Because a friend and I are collaborating on this as we learn the ropes. We both record ES, then combine the data, then backtest/optimize using the single combined dataset. make sense? -greg On Mon, Dec 13, 2010 at 7:29 PM, ShaggsTheStud <[email protected]>wrote: > If you want a text editor that can handle huge files, for windows, there is > TextPad. It doesn't choke at all. > > On Sun, Dec 12, 2010 at 3:45 PM, Gregory Scott Burd <[email protected]>wrote: > >> This weekend I had need of a tool to combine a few log files of ES GLOBEX >> futures data gathered by JBookTrader into a single log file. Finding none I >> wrote a short python script to do just that. >> https://gist.github.com/738469 >> >> I'm curious if anyone wants to contribute files so that we can combine ES >> data logs into one more complete log so that we can all benefit. >> >> Any takers? >> >> -greg >> >> -- >> You received this message because you are subscribed to the Google Groups >> "JBookTrader" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<jbooktrader%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/jbooktrader?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "JBookTrader" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<jbooktrader%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/jbooktrader?hl=en. > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
