Hello list,
I think I found a way to enhance splitter memory usage.
I have now a version on my pc that is
- a bit faster (at least for my largest test data "germany.osm.pbf" it saves
~29% runtime: 725 sec. instead of 1009 sec (AMD Dualcode 5050e+4GB+Win-XP)
- able to handle node ids up to 137.438.953.471L (= 2^37-1) (from both xml
and osm.pbf as far as I can say)
- still less memory consuming. esp. for data with very high node ids
I achived that by using a long for the id values and by replacing the two
ArrayLists in SparseInt2ShortMapInline by an "Int2ObjectOpenHashMap"
(fastutil) so that the HashMap stores objects like this:
class Storer {
private long chunkmask;
private short [] chunk;
}
The meaning of these two fields is the same as in the existing code in
SparseInt2ShortMapInline.
I did not test the change with a whole planet because my machine is too slow
for that and I have problems with the download of the large file.
Since the new code needs a different fastutil.jar, I can't simply add a
patch. If anybody is interested in testing with whole planet or reviewing my
code, what is the proprosed way to do this?
Ciao,
Gerd
--
View this message in context:
http://gis.638310.n2.nabble.com/splitter-memory-usage-tp6935688p6945302.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
_______________________________________________
mkgmap-dev mailing list
[email protected]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
