Peng Yu wrote:
The file size of a wig file can be very large (GB). Most tasks on this file format does not need the parser to save all the lines read from the file in the memory to produce the parsing result. I'm wondering if pyparsing is capable of parsing large wig files by keeping only minimum required information in the memory.
I cannot recommend pyparsing for large amounts of text. Even before you hit memory limits, you will run into the problem that pyparsing runs many functions for each character of text. Python function calls are expensive.
Since the format is line-oriented, one option is to use pyparsing or other parser to handle the track definition lines and just str.split(), float() and int() for the data lines.
-- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
