Hi, I am using the IndexedObjectStore for storing and then accessing large amounts of data - around 300 millions objects. In the maps I am storing the Long's as the key and simple objects with few properties as values. The maps are backed up in the file system by 5 files with size ranging from 3 GB to 11 GB.
When I start accessing the data from the collections, I am experiencing quite slow performance - just to enumerate all objects in the collection it takes around 15 hours on the 7200 rpm hard drive, with 10G of memory available to java vm. The java vm runs in the server mode. I can see that the machine resources - CPU, hard drives are utilized to a very small amount, the respective performance counters are close to minimal. I have tried to perform multi-threaded reads - in each thread I create separate indexed store readers. But the result is similar - the benefit is very small. Could anyone give me any suggestions how I can improve the data access and utilize the machine resources more efficiently? Yours sincerely, Dennis Frostlander, P.S. on the related topic, I noticed that when the java process runs in the debug mode and the debugger is attached (either intellij idea or eclipse), the read operations are a magnitude slower. Not really sure why though... Yours sincerely, Dennis Frostlander,
_______________________________________________ osmosis-dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/osmosis-dev
