Wow, those results are compelling. Indeed I thought before of basing the streaming parser on a pull parser... seemed more natural then the hack of creating a separate parsing thread.
A 100 features taking up 606 MB seems out of hand though. Are you using the xpath streaming method? That message is grossly inefficient, but i thought the class based and name based streaming methods were better. Perhaps not though. Anyways, i will check out the code you submitted. What do you think about tieing up your simple parser to the binding stuff? Gabriel Roldán wrote: > Hi Justin, > > I've been working on getting the nsdi test server data rendering in udig by > using the StreamingParser in the WFS DataStore, and got it. > > Yet, its leading to OOM errors on the first try. I have yet to run it inside > a > profiler, though in the meantime I did a quick (simple) feature parser based > on xml pull to get something done while we tackle the StreamingParser > problems. > > Also, to assess this problem I've created a couple normal java applications > in > the tests folder so its easy to be run, as I imagine you don't have to bother > with setting up a udig trunk development environment. > > So, in the gt-wfs module, there are two test classes, > XmlSimpleFeatureParserTest and StreamingParserFeatureReaderTest, that can be > run as normal java applications through their main methods. They exercise the > parsing of the same GetFeature response from the nsdi server, and are tests > for the two stratagy objects used for that purpose. > > A sample output of running them as java apps (not unit tests) is bellow, > which > shows more or less the memory problem. Yet, in the long run I'll need, or > rather would prefer, the StreamingParser to function properly, so we don't > have yet another parsing technology around. If you have any clue about what > can be happening that'd be cool. > > Sample output from the bogobenchmark apps: > XmlPull Feature parser: > Fetched 100 features in 111654ms. (avg. 1116ms/feature) Mem. used: 7MB. > > StreamingParser > Fetched 100 features in 112481ms. (avg. 1124ms/feature) Mem. used: 606MB. > > (the request is setting maxFeatures=100 or it becomes an endless wait, but > the > OOM happens with enough patience) > > Gabriel > > !DSPAM:4007,47b33b66162485210051143! > -- Justin Deoliveira The Open Planning Project [EMAIL PROTECTED] ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Geotools-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
