>>>>> "FV" == VERHOEF Frans <[EMAIL PROTECTED]> writes:
FV> Hi, I am having a problem parsing huge blast
FV> results. Basically I am parsing the blast results pretty much
FV> the same way as in "Biojava in Anger", with as only difference
FV> that I use the setModeLazy() of the BlastLikeSAXParser, since
FV> I am using NCBI Blast version 2.2.4 and that version is not
FV> recognised by the parser yet.
Using blast 2.2.4 or 2.2.6 is safe in lazy mode - diffs show only
minor whitespace changes in the format.
FV> Besides that the only difference lays in the things I do with
FV> the data.
This is likely to be the cause of the problem. See below.
FV> The problem is that when I parse a blast result that is a few
FV> hundred MB, for example 300MB, the java application is
FV> ballooning up to around 1.6GB of memory. Sometimes the
FV> application even crashes because I only have got 2GB to play
FV> with.
The parser uses an event driven framework which is designed to handle
very big data - it will handle multi-GB reports. However, if you
create many fine-grained objects for every element of every report you
will quickly run out of resources.
FV> Does anyone know what's causing this? Is it because I set the
FV> lazy mode? Is there any way to work around it?
Either you need to think about which elements of the report you are
interested in and build a filter which captures those events,
discarding the rest. See the demos/ssbind package for an example by
Matthew. Or if you really need all those objects then you should look
at allowing them to be garbage-collected as soon as possible.
It is possible that there is a bug somewhere, but without any seeing
any code it isn't possible to say much more. If you need more help,
post a short (working) piece of code illustrating the problem and we
will do our best.
hth
Keith
--
- Keith James <[EMAIL PROTECTED]> Microarray Facility, Team 65 -
- The Wellcome Trust Sanger Institute, Hinxton, Cambridge, UK -
_______________________________________________
Biojava-l mailing list - [EMAIL PROTECTED]
http://biojava.org/mailman/listinfo/biojava-l
