On 29.08.2011 20:52, WanMil wrote:
1. I noticed that (at least for my style-file) 7500MB of RAM available
does not suffice to compile a map of Europe including index! However if
I recreate the index in another go, then it is no problem.
How comes? The index creation seems to be completly independent of the
maptiles creation. Is mkgmap leaving stuff inside the memory, that is in
reality not needed anymore?
I think you have listed your style and your mkgmap parameters on your
website. Can you give a link to them? That makes it easier to find the
problem.
Subversion
* Path: https://svn.origo.ethz.ch/openmtbmap/
* Web SVN: http://svn.origo.ethz.ch/wsvn/openmtbmap/
* ViewVC: http://svn.origo.ethz.ch/viewvc/openmtbmap/
* Nightly SVN Dump: http://svn.origo.ethz.ch/dump/openmtbmap/latest.tar.gz
And here is the specific options on full creation (crashes low on java
heap space, passes if Xmx11000M is given, but I only have 12GB RAM on a
testmachine on my build sever there is only about 7800M before it
seriously slows down -- this is running against Europe extract from
geofabrik):
c:\OpenMTBMap\maps>start /low /b /wait java -ea -jar -Xmx7500M
c:\openmtbmap\mkgmap.jar --max-jobs=4
--generate-sea=extend-sea-sectors,close-gaps=6000,floodblocker,fbgap=60,fbthres=200,fbratio=0.6
--latin1 "--style-file=c:\openmtbmap\new4" --reduce-point-density=4
--nsis --index --transparent --adjust-turn-headings --add-pois-to-areas
--ignore-maxspeeds --x-reduce-point-density-polygon=8
--link-pois-to-ways --ignore-turn-restrictions --min-size-polygon=15
--remove-short-arcs=4 --description=openmtbmap_eu --merge-l
ines --location-autofill=1 --route --country-abbr=eu
--country-name=europe --mapname=65500000 --family-id=6550
--product-id=1 --series-name=openmtbmap_europe_29.08.2011
--family-name=mtbmap_eu_29.08.2011 --tdbfile --overview-mapname=mapset
--keep-going --area-name="europe_29.08.2011_openmtbmap.org" -c
c:\openmtbmap\maps\template.europe 1>NUL
versus the options for index only creation (works well, I can even
reduce Xmx to about 4500M):
c:\OpenMTBMap\maps>start /low /b /wait java -ea -jar -Xmx7500M
c:\openmtbmap\mkgmap.jar --nsis --index --transparent
--description=openmtbmap_eu --location-autofill=1 --country-abbr=eu
--country-name=europe --mapname=65500000 --family-id=6550
--product-id=1 --series-name=openmtbmap_europe_29.08.2011
--family-name=mtbmap_eu_29.08.2011
--tdbfile --overview-mapname=mapset --keep-going
--area-name="europe_29.08.2011_openmtbmap.org" 6550*.img
And important: Do you use any patches?
yes listed here:
http://svn.origo.ethz.ch/wsvn/openmtbmap/mkgmap_patches.patch
but they shouldn't affect the index creation. Maybe it's again an issue
of assigning names like ft to unnamed footways. However there must still
be some memory leak if creating the index standalone it needs only about
50% of memory. Actually it would be really great if the index creation
could be lighter on memory, maybe using a temporary file for buffer?
WanMil
2. Also as running out of memory primarily happens on index creation, it
would be great if mkgmap writes out the index under a temporary name,
and once it has passed correctly, renames the file. Else it is not
possible to easily assess in a bash/batch script whether mkgmap
completed the map correctly or not (and hence one uploads broken maps if
everything automated). Or at least I don't see any way to find out
whether it passed correctly or not.
_______________________________________________
mkgmap-dev mailing list
[email protected]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
