Hi, On 12/16/10 10:13, Stefan de Konink wrote:
I think Mitja did this in the GSoC program this summer (the cutout thing). It would be valuable to figure out a way to do 'proper' full ways and relations in a low memory setting, but probably I/O itself is always an issue too.
I'd be interested to hear from Mitja about his solution (even better, find it in SVN).
I can imagine that you create a cutout of the nodes first, quicksort it in a mmap-file and then run it over the ways table. Storing a mmap file with only the way ids,quicksort on it and walk over the relations part. Now a second run would grab the missing ways, and the remaining nodes.
If you allow multiple passes then no need to do sparse stuff - just use bit vectors to flag used nodes, ways, relations. Takes 200 MB of RAM at max and no sparse-set/sorting/searching overhead.
Of course the multiple passes is what kills performance, especially if you have to bunzip2/xml-parse each time.
In addition, it would be cool for a history excerpt to always extract *all* versions of an object even if only some of them actually match the criterion. That could be trivially added if the input is sorted - simply start a buffer at version 1 and dump it when the next object starts.
Bye Frederik _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
