Hello Scott, How do you keep track of what bboxs each entity belongs to ?
I'm not really asking a question, I'm just saying that I found a way to reduce the memory requirement for that considerably. Instead of a bit per bbox per entry, I store only 16 bits or 32 bits per entry. Here is the source. http://trac.openstreetmap.org/browser/applications/rendering/gosmore/bboxSplit.cpp?rev=24484 Ask if you have any questions. On Wed, Dec 1, 2010 at 2:59 PM, Scott Crosby <[email protected]> wrote: > I've been working on making various improvements to the mkgmap > splitter. It was originally intended to split a map into smaller > pieces which could then be converted into Garmin format and artifacts > of that original purpose continue to exist. (Eg, its coordinate system > for representing the areas.) > > However, I've committed several improvements to crosby_integration > branch in mkgmap SVN that massively increase its scalability and may > make it more useful to a general audience: > > On a 8gb core i7 machine, the new version can: > Split the whole planet into 1200 tiles in about an hour (from pbf > to osm.gz format) in a single pass. > Split the whole planet into 20000 tiles in about 4 hours over 3 passes. > > There are some limitations, both in the implementation, and design. > > Due to inherent serialization of the internal tracking tables, it > only uses a few CPU cores. > > The code uses bounding boxes, not bounding polygons. In the > original purpose of generating mkgmap tiles, tiles are square there > are no polygon tests. Polygon support is straightforward to add, but a > fileformat must be defined to specify bounding polygons instead of > bounding boxes. > > At thousands of tiles being output concurrently in a pass, memory > consumption in output buffering can be significant. > > The process assumes that the file is sorted with nodes first, then > ways, then relations. Efficiency degrades if they are not further > sorted based on ID. > > Ways are not 'completed', A way is included if any node of it is > within the bounding box. Other nodes in the way are not included > unless they too lie inside the bounding box. (So, typically, the > bounding box is expanded to include an overlap region.) > > All metadata is dropped. > > The code hasn't been generalized to overcome these limitations, but > might be useful to those creating lots of excerpts or breaking maps up > by continent, nation, state, or county. As I don't expect to have much > time to work on the code, I mention it now. > > Scott > > _______________________________________________ > dev mailing list > [email protected] > http://lists.openstreetmap.org/listinfo/dev >
_______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
