Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
2010/11/17 Frederik Ramm frede...@remote.org: https://bugs.launchpad.net/ubuntu/+source/bzip2/+bug/621130 I could reproduce the bug here but only when piping uncompressed stuff through bzcat -f, which is a rather stupid thing to do anyway. Whenever I piped compressed stuff into it, it worked as expected. I discovered this bug when I tried to import (compressed) srtm-data I had converted with srtm2osm into my pgdb with osm2pgsql (I got the error file is empty because the first line wasn't uncompressed correctly). I resolved by decompressing and recompressing it with gzip, so I am quite sure the osm-file was OK (file, more and tail also confirmed this). I could reproduce this bug also by simply compressing a whatever textfile of 2-3 lines and then reading it with bzcat (the first line got awful). cheers, Martin ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
On Mon, 2010-11-15 at 22:28 +0100, Stefan de Konink wrote: But osm2pgsql reports: Reading in file: - Processing: Node(832188k) Way(7341k) Relation(0k)Entity: line 1275668752: parser error : Specification mandate value for attribute v tag k=source v So dear XML lovers, what went wrong with with the planet generation? Last night I successfully ran this planet file through the process which generates the coastline shapefiles without seeing any error. The initial osm2coast processing step uses almost identical XML parsing code to that used by osm2pgsql so I don't believe there is any issue with the format of this planet file. Are the errors you see consistent between runs? Could you try running osm2pgsql with the null output mode --output null and see whether that reports any problems parsing the data? Jon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
On Tue, 16 Nov 2010, Jon Burgess wrote: Are the errors you see consistent between runs? Basically 'still' importing now. I can possitively confirm that this is because of 'out of memory' situations. You may wonder if instead of using memory osm2pgsql shouldn't go for a mmaped areas (so instant swap). I guess either some operations are not checked for null pointers, or something else fishy is going on. Stefan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
Jon, Jon Burgess wrote: What I meant was that the bug I referenced appears to occur in the recent Karmic and Lucid code. I'll include it again since you chopped it out in your reply: https://bugs.launchpad.net/ubuntu/+source/bzip2/+bug/621130 I could reproduce the bug here but only when piping uncompressed stuff through bzcat -f, which is a rather stupid thing to do anyway. Whenever I piped compressed stuff into it, it worked as expected. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Our beloved XML format doesn't parse; planet 11-11
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, After 4 unsuccesfull in osm2pgsql runs, and today after md5 validation. I am pretty sure that the file that I have downloaded is complete. ea628ba9912e6d33ef65d35b7cbe6823 planet-latest.osm.bz2 But osm2pgsql reports: Reading in file: - Processing: Node(832188k) Way(7341k) Relation(0k)Entity: line 1275668752: parser error : Specification mandate value for attribute v tag k=source v ^ Entity: line 1275668752: parser error : attributes construct error tag k=source v ^ Entity: line 1275668752: parser error : Couldn't find end of Start Tag tag tag k=source v ^ - - : failed to parse So dear XML lovers, what went wrong with with the planet generation? Stefan -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkzhpekACgkQYH1+F2Rqwn2oZQCeP2QF8yQaeWq5iDkgEVv3RTTg gMYAnigK4moNA6W7AUOJ7A/NDJnwR/sQ =4ytH -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
On Mon, 2010-11-15 at 22:28 +0100, Stefan de Konink wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, After 4 unsuccesfull in osm2pgsql runs, and today after md5 validation. I am pretty sure that the file that I have downloaded is complete. ea628ba9912e6d33ef65d35b7cbe6823 planet-latest.osm.bz2 But osm2pgsql reports: Reading in file: - Processing: Node(832188k) Way(7341k) Relation(0k)Entity: line 1275668752: parser error : Specification mandate value for attribute v tag k=source v ^ Entity: line 1275668752: parser error : attributes construct error tag k=source v ^ Entity: line 1275668752: parser error : Couldn't find end of Start Tag tag tag k=source v ^ - - : failed to parse So dear XML lovers, what went wrong with with the planet generation? Have you changed anything recently? A week or two ago someone pointed me at a possible issue with the current bzip2 code in Ubuntu when decompressing to a pipe: https://bugs.launchpad.net/ubuntu/+source/bzip2/+bug/621130 I'll try parsing the file but it'll take several hours to run. Jon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Op 15-11-10 23:38, Jon Burgess schreef: Have you changed anything recently? Like what, it was a fresh download, and I have never used PostGIS on my system. Stefan -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkzhtyIACgkQYH1+F2Rqwn29nQCdE22v9PoeTYQFgrUW30BRG+vr lygAn2ppZP3lkDj3bCIBo1bAzTz2tvnm =NYm5 -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Our beloved XML format doesn't parse; planet 11-11
On Mon, 2010-11-15 at 23:41 +0100, Stefan de Konink wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Op 15-11-10 23:38, Jon Burgess schreef: Have you changed anything recently? Like what, it was a fresh download, and I have never used PostGIS on my system. What I meant was that the bug I referenced appears to occur in the recent Karmic and Lucid code. I'll include it again since you chopped it out in your reply: https://bugs.launchpad.net/ubuntu/+source/bzip2/+bug/621130 I was wondering whether you had successfully decompressed older planet files but are now seeing this problem after recently upgrading your OS and downloading a new planet file. The issue might possibly be with the system bzip2 library and not the planet file. Jon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
