I used max-nodes=2 300 000
so it ends up with 143 tiles.
255 didn't give significant improvements (maybe 20 seconds).
I will probably run splitter now with 1024 max-areas. Hope 8.4GB heap
will be enough for it..
It would probably be good if the splitter assumes +50% or whatever you
seem reasonable for max memory on areas for the first pass..
I will give 2048 max-areas a try too, just to see if I can split Europe
in 1 pass. If it works, I'll stick with it. Still need to wait for the
2GB additional Ram for my server though...
On 13.12.2012 10:12, Gerd Petermann wrote:
Hi Felix,
thanks for the tests. I am quite happy with your results.
In the meantime I have fixed some memory leaks.
Could you please tell whether you use the default of max-areas=255 or
a higher value?
The reason why I ask:
When spliting france with max-nodes=1000000 you'll get > 255 areas,
and 255 is the default for max-areas.
Splitter uses the max-areas value for the problem-list-generator AND
for the final distribution pass.
Both use the same data structures to store informations, but the
distribution pass requires
additional heap for the buffers allocated for each open output file,
and with
a large number of areas and pbf or o5m output these additional buffers
can
require > 1 GB.
I try to find a good algo to decide how many passes are needed in the
problem-list generator. I am pretty sure that most users have enough
heap to run this phase with a single read pass.
Gerd
> Date: Tue, 11 Dec 2012 14:23:57 +0100
> From: [email protected]
> To: [email protected]
> Subject: Re: [mkgmap-dev] splitter r254
>
> Well, I just ran a couple of tests with it. On france.osm.pbf splitting
> takes 20minutes with --keep-complete, vs 12 minutes with
--overlap=4000.
> On the other hand, mkgmap needs about 4 minutes less (albeit consuming
> more memory). Currently on France and a few other countries, the added
> memory consumption means, I have to use 3 processes for mkgmap instead
> of 4 (the comparison times were with 3) - I'll spec up to 10GB RAM,
> which should then be enough for 4 processes parallel. Overall using
> keep-complete instead of overlap=4000, will addup a bit of time, but
not
> much - as mkgmap runs a bit quicker (of course compared to
overlap=2000,
> mkgmap runs not much quicker, as the bigger the overlap, the bigger the
> time increase for mkgmap)
>
> For other countries results were similar.
>
> The new algo to avoid very small tiles is working great. France.osm.pbf
> from Geofabrik should lock up / BSOD any Garmin GPS devices any more.
> _______________________________________________
> mkgmap-dev mailing list
> [email protected]
> http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
--
keep on biking and discovering new trails
Felix
openmtbmap.org & www.velomap.org
_______________________________________________
mkgmap-dev mailing list
[email protected]
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
