On Mi, Apr 20, 2016 at 06:59:35 +0200, Christoph Hormann wrote: > On Wednesday 20 April 2016, Jochen Topf wrote: > > > > Adding node locations to ways is a pain (and needs more than 32 GB > > RAM these days for a full planet if you want to get it done in > > reasonable time). So I experimented with adding the node locations to > > the ways once and storing it in the OSM files. Works for XML, PBF, > > and OPL. See my blog post at > > https://blog.jochentopf.com/2016-04-20-node-locations-on-ways.html > > for the full story. > > From perspective of database design this of course introduces redundancy > in the data which in turn bears the risk of introducing > inconsistencies. For storing data temporarily during processing this > is not generally a big deal because the processes writing and reading > this data are well defined but as a general data exchange and storage > format this would be highly problematic - depending on how a program > reads the file it will potentially get different results if there are > inner inconsistencies in the file.
Well, there are always ways of screwing up the data... > On a more general note: for the purpose of storing the OSM planet data > in a form that is efficient to use and update wouldn't it make sense to > store the node locations in a flat file separately from the rest > instead of storing everything in a dynamically structured format like > XML or PBF that can essentially only be read sequentially? Sure. That would also make sense for some application. But doesn't solve the problem I am trying to solve here, that is allowing you to work with OSM data without much RAM. > Another thing: you say you throw away untagged nodes that are not member > of any way - don't forget nodes can also be relation members. I don't > know any practical case where an untagged node is a relation member but > in principle this is possible and could make sense. Thats true. I thought about adding the node location also to the relation member, but, as you say, in most cases you'll need the tags of the member, too, which would make this a whole lot more complicated. So I can't help with that use case currently. Jochen -- Jochen Topf [email protected] http://www.jochentopf.com/ +49-351-31778688 _______________________________________________ dev mailing list [email protected] https://lists.openstreetmap.org/listinfo/dev
