Ashley,
I took a quick look at your example in a profiler. I don't have the
same kml file to compare against but I was able to run your sample
program on a 34k feature kml file in 14 seconds. Obviously this is
apples to oranges but its a place to start.
Roughly 9 seconds were spent in the StreamingParser. If you don't want
to use a profiler you could just comment out the writeFeature line and
get a rough idea how much that is adding to the total.
The StreamingParser code doesn't have any obvious time wasters.
About 1/3 of the time was spent writing the SimpleFeatures with
FeatureJSON. I didn't look at the writeFeature method, it might be
possible to speed that up but I doubt it.
Your example has a big loop in it. It looks like we can't run each
iteration independently and we can't speed up the components of the
loop. But we can turn the loop iteration into a pipeline and run each
part of the loop in a separate thread.
In your example you don't start parsing the next SimpleFeature until
after you've finished writing the previous SimpleFeature to disk. Split
your read/write loop into a producer-consumer pattern. In one thread
parses the input file into SimpleFeatures and the other thread writes
those out as JSON. Stick a BlockingQueue in between the two threads so
that you won't fill up memory with SimpleFeatures.
That turned 14 seconds into 10 seconds.
-Ryan
On 4/28/2011 6:42 AM, Justin Deoliveira wrote:
Hi Ashley,
Unfortunately not really as streaming GML parsing options are limited.
So I guess the answer would be to develop your own feature parser. Or
try to profile the existing one to figure out how/if performance can
be improved.
-Jusitn
On Wed, Apr 27, 2011 at 10:25 AM, mortac8 <[email protected]
<mailto:[email protected]>> wrote:
Any tips on how to improve its performance? By using a different
XML parser
or would you improve the performance in some other way? Maybe I
could help.
Thanks again!
Ashley
--
View this message in context:
http://osgeo-org.1803224.n2.nabble.com/Large-KML-file-to-GeoJSON-performance-question-tp6306341p6310159.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.
------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
Geotools-gt2-users mailing list
[email protected]
<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
Geotools-gt2-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
Geotools-gt2-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
