Op 12-Jan-12 21:10, Yang schreef: > thanks, I did utilize the useMemoryMappedBuffer option. let me dig up > the first one. > > by "copy all of the shapefile into a DataStore or feature store that > uses a memory stucture " do you mean something like "MemoryDataStore" > ( > http://udig.refractions.net/files/docs/api-geotools/org/geotools/data/memory/MemoryDataStore.html > ) ? so basically I need to iterate through all the features in my > existing ShapeFileDataStore, and insert them into MemoryDataStore, and > hopefully the later queries would be faster , right? sounds like a > possible approach.
yes, this would save using file io. i've never used memoryDataStore and the docs (http://docs.geotools.org/latest/userguide/library/data/memory.html) say it may be slow, so you may want to cook up one yourself. Maybe one of the SimpleFeatureCollection implementations together with subCollection(Filter) may also fit your need. > actually I did a later test and put all my indexedShapeFileDataStore > on a tmpfs mount (basically memory disk), and surprisingly the > performance did not change much from direct disk access. most of the > time is still spent on reading file buffers and opening index file > readers this opening and closing of file handles is probably the most costly part. I try to steer away from shapefiles whenever I can. maybe one of the devs has something to say about this, but it's a bit difficult without you code. > > Yang > > On Thu, Jan 12, 2012 at 11:37 AM, Mark Prins<[email protected]> wrote: >> Op 12-Jan-12 01:36, Yang schreef: >>> my code looks up features from a IndexedShapeFileDataStore >>> >>> basically: >>> >>> for(Point p : given points ) { >>> dataStore.getFeature(query(p)); >>> } >>> >>> >>> I can see from "top" command that the RES memory consumption is only 40MB, >>> I have about 8GB on the box, and have allocated -Xmx4000M to jvm. >>> is there any way I could let geotools load more content into memory, >>> so hopefully it would query faster? >> >> you could try to copy all of the shapefile into a DataStore or feature >> store that uses a memory stucture such as eg. a hashtable to create an >> in memory cache of the data, of course this will only work reliably in a >> read-only scenario. it depends on you data if it's worth the tradeoff >> with longer startup time. >> >> btw. don't the "cacheMemoryMaps" and "useMemoryMappedBuffer" options on >> IndexedShapeFileDataStore() do anything? >> >> >> ------------------------------------------------------------------------------ >> RSA(R) Conference 2012 >> Mar 27 - Feb 2 >> Save $400 by Jan. 27 >> Register now! >> http://p.sf.net/sfu/rsa-sfdev2dev2 >> _______________________________________________ >> Geotools-gt2-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users ------------------------------------------------------------------------------ RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 _______________________________________________ Geotools-gt2-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
