On 2013-02-13 18:39, monarch_dodra wrote:
In any case, I am now parsing the 6Gig packed into 1.5Gig in about 53 seconds (down from 61). I also tried doing a dual-threaded approach (1 thread to unzip, 1 thread to parse), but again, the actual *parse* phase is so ridiculously fast, that it changes *nothing* to the final result.
Great. Performance aside, we didn't talk much about how this data can be useful - should it only be read sequentially forward or both ways, would there be a need to place some markers or slice the sequence, etc. Our small test case was only about counting nucleotides, so reading order and possibility of further processing was irrelevant.
Mr.Bio, what usage cases you'll be interested in, other than those counters?
