Re: [OSM-talk] [OSM-dev] Managing large XML files aka osm.xml
Hi Artem, This is really useful. I think it'll be especially handy for me if our main osm.xml file uses this - at the moment it's quite a lengthy procedure to resync the cycle map with the base osm.xml, since I'm trying to keep all of the roads rules the same bar the colour. It's hard to see what's going on when diff'ing, but this should make it easier. Have you any thoughts on using this for e.g. tunnels? i.e. where you have a large number of style rules (e.g. roads) and you want to say do all of that again, but this time dotted when adding 'and tunnel = yes' to the selector, without having to verbosely repeat all the road rules again? I'm starting to find that with the cycle stuff, with proposed routes etc, that the rules are fairly repetitive. There might be a clever way to implement this concept using these entities, but was wondering if you'd thought it through already for me. Cheers, Andy On Jan 2, 2008 3:50 PM, Artem Pavlenko [EMAIL PROTECTED] wrote: Hello all! David Siegel implemented support for libxml2 parser in Mapnik, check it out : http://mapnik.org/news/2008/jan/02/managing_large_xml/ http://trac.mapnik.org/wiki/ManagingLargeXmlFiles Happy New Year! Artem ___ dev mailing list [EMAIL PROTECTED] http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] Managing large XML files aka osm.xml
On Jan 2, 2008 4:37 PM, Stefan de Konink [EMAIL PROTECTED] wrote: On Wed, 2 Jan 2008, Artem Pavlenko wrote: David Siegel implemented support for libxml2 parser in Mapnik, check it out : I was able to load the entiere planet.osm into MonetDB4 (XML), but incompare to the SQL version it is not optimal. And one may wonder why he wants to use an text representation instead of a processed binary! For plain XML processing people should probably stick with MSXML for the time being. The performance of it is probably the best one out there. We did some XSLT processing based on the [EMAIL PROTECTED] stuff, and it is not funny that an 8 times as powerful machine gets beaten. I wonder how much time it will take for each run on an 8GB machine processing the current planet.osm (61GB). This isn't to do with xml parsing of the planet.osm. This is to do with using entities within the mapnik rules file, e.g. osm.xml from http://svn.openstreetmap.org/applications/rendering/mapnik/ which in itself is lines long and very verbose. Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] Managing large XML files aka osm.xml
On 2 Jan 2008, at 16:37, Stefan de Konink wrote: On Wed, 2 Jan 2008, Artem Pavlenko wrote: David Siegel implemented support for libxml2 parser in Mapnik, check it out : I was able to load the entiere planet.osm into MonetDB4 (XML), but incompare to the SQL version it is not optimal. And one may wonder why he wants to use an text representation instead of a processed binary! Sorry, if it was not clear. Mapnik has got three xml parsers to choose from (at compile time) : tinyxml, spirit and now libxml2. Only libxml2 based supports advanced features like entities. Mapnik is using XML parsers to load data/styles definition file, not actual osm data. For plain XML processing people should probably stick with MSXML for the time being. The performance of it is probably the best one out there. I would doubt it. BTW, all three parsers mentioned above run happily on win32. We did some XSLT processing based on the [EMAIL PROTECTED] stuff, and it is not funny that an 8 times as powerful machine gets beaten. I wonder how much time it will take for each run on an 8GB machine processing the current planet.osm (61GB). I understand but I never decompress planet : : osm2pgsql planet.osm.bz2 Stefan Artem ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] [OSM-dev] Managing large XML files aka osm.xml
On 2 Jan 2008, at 16:46, Andy Allan wrote: Hi Artem, This is really useful. I think it'll be especially handy for me if our main osm.xml file uses this - at the moment it's quite a lengthy procedure to resync the cycle map with the base osm.xml, since I'm trying to keep all of the roads rules the same bar the colour. It's hard to see what's going on when diff'ing, but this should make it easier. Absolutely! Have you any thoughts on using this for e.g. tunnels? i.e. where you have a large number of style rules (e.g. roads) and you want to say do all of that again, but this time dotted when adding 'and tunnel = yes' to the selector, without having to verbosely repeat all the road rules again? I'm starting to find that with the cycle stuff, with proposed routes etc, that the rules are fairly repetitive. There might be a clever way to implement this concept using these entities, but was wondering if you'd thought it through already for me. No, haven't thought about this, yet:) But it looks like entities might be useful in this situation, too. I guess we need to try and see. Cheers, Andy Artem On Jan 2, 2008 3:50 PM, Artem Pavlenko [EMAIL PROTECTED] wrote: Hello all! David Siegel implemented support for libxml2 parser in Mapnik, check it out : http://mapnik.org/news/2008/jan/02/managing_large_xml/ http://trac.mapnik.org/wiki/ManagingLargeXmlFiles Happy New Year! Artem ___ dev mailing list [EMAIL PROTECTED] http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
