Re: [OSM-dev] Guys I trusted you, I removed my checks...
Paul Johnson wrote: Stefan de Konink wrote: but again this trust was misplaced. No, it is not Potlatch 0.9a, it cannot be Potlatch is the perfect user tool and introduction to OSM. It must be API 0.6 that didn't solve all our problems, as was promised. Instead but it opened the gates of hell, a parallel universe within OSM...[/melodramatic] [start fanfare tune + brass] http://api.openstreetmap.org/api/0.6/way/8115655/full http://api.openstreetmap.org/api/0.6/node/255483811/history I probably have to blame myself; I was a fool to speed up my converter by removing the consistency checks. But since when do we support more than -180/+180 ? My understanding is that there's nothing intrinsically tying us to ±90° by ±180°, based on the responses I got on #osm when asking about the possibility of mapping something with a geometry grossly different from Earth's (ie, Second Life). On Earth, at least, anything outside of the ±180x90° range is likely subject to standard trigonometry in terms of placement, assuming it's not rejected. As for placement on the map: Mapnik puts the Norhtpole on lat/lon 0,0 (have a look). Maybe that's the 5th dimension translation of it, so who knows what lat/lon larger than 90/180 does. Regards, Maarten ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] merkaator debuild fails ...
Could someone have a look at this please ... Complete log is at: http://www.gpsdrive.de/build_cluster/merkaartor/debian-squeeze-64/debuild.log.shtml tmp/obj_debug/qrc_MYahooBackground.o -L/usr/lib -lQtGui -lQtCore -lpthread mv -f libMYahooBackgroundPlugin.so ../../../binaries/bin/plugins/background/ make[4]: Leaving directory `/home/tweety/svn.openstreetmap.org/applications/editors/merkaartor/plugins/background/MYahooBackground' cd FrenchCadastre/ /usr/share/qt4/bin/qmake FrenchCadastre.pro -unix -o Makefile Cannot find file: FrenchCadastre.pro. make[3]: *** [FrenchCadastre//Makefile] Error 2 make[3]: Leaving directory `/home/tweety/svn.openstreetmap.org/applications/editors/merkaartor/plugins/background' make[2]: *** [sub-background-make_default] Error 2 make[2]: Leaving directory `/home/tweety/svn.openstreetmap.org/applications/editors/merkaartor/plugins' make[1]: *** [sub-plugins-make_default] Error 2 make[1]: Leaving directory `/home/tweety/svn.openstreetmap.org/applications/editors/merkaartor' make: *** [build-stamp] Error 2 debuild: fatal error at line 1306: couldn't exec fakeroot debian/rules: -- Jörg (Germany, Tettnang) http://www.ostertag.name/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] merkaator debuild fails ...
Solved. Do not forget to rerun qmake. Please not that there is a merkaartor-specific list at http://lists.openstreetmap.org/listinfo/merkaartor;. Dev is rather for the core openstreetmap. Regards - Chris - 2009/5/19 Joerg Ostertag (OSM Tettnang/Germany) openstreet...@ostertag.name Could someone have a look at this please ... Complete log is at: http://www.gpsdrive.de/build_cluster/merkaartor/debian-squeeze-64/debuild.log.shtml tmp/obj_debug/qrc_MYahooBackground.o -L/usr/lib -lQtGui -lQtCore -lpthread mv -f libMYahooBackgroundPlugin.so ../../../binaries/bin/plugins/background/ make[4]: Leaving directory `/home/tweety/ svn.openstreetmap.org/applications/editors/merkaartor/plugins/background/MYahooBackground ' cd FrenchCadastre/ /usr/share/qt4/bin/qmake FrenchCadastre.pro -unix -o Makefile Cannot find file: FrenchCadastre.pro. make[3]: *** [FrenchCadastre//Makefile] Error 2 make[3]: Leaving directory `/home/tweety/ svn.openstreetmap.org/applications/editors/merkaartor/plugins/background' make[2]: *** [sub-background-make_default] Error 2 make[2]: Leaving directory `/home/tweety/ svn.openstreetmap.org/applications/editors/merkaartor/plugins' make[1]: *** [sub-plugins-make_default] Error 2 make[1]: Leaving directory `/home/tweety/svn.openstreetmap.org/applications/editors/merkaartor' make: *** [build-stamp] Error 2 debuild: fatal error at line 1306: couldn't exec fakeroot debian/rules: -- Jörg (Germany, Tettnang) http://www.ostertag.name/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Guys I trusted you, I removed my checks...
2009/5/18 Stefan de Konink ste...@konink.de: ...but again this trust was misplaced. No, it is not Potlatch 0.9a, it cannot be Potlatch is the perfect user tool and introduction to OSM. It must be API 0.6 that didn't solve all our problems, as was promised. Instead but it opened the gates of hell, a parallel universe within OSM...[/melodramatic] [start fanfare tune + brass] http://api.openstreetmap.org/api/0.6/way/8115655/full http://api.openstreetmap.org/api/0.6/node/255483811/history I probably have to blame myself; I was a fool to speed up my converter by removing the consistency checks. But since when do we support more than -180/+180 ? Stefan Current API code has limits for the standard sane coordinate limits, and it always has done as far as I can remember (back to mid 0.4, but I wasn't coding then..) It probably just hadn't come to anyone's attention that we did have bad coordinates in the database from some source, and nobody bothered to run a consistancy check on them... -- Regards, Thomas Wood (Edgemaster) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Guys I trusted you, I removed my checks...
-Original Message- From: dev-boun...@openstreetmap.org [mailto:dev- boun...@openstreetmap.org] On Behalf Of Stefan de Konink Sent: 18 May 2009 23:57 To: OSM-Dev Openstreetmap Subject: [OSM-dev] Guys I trusted you, I removed my checks... ...but again this trust was misplaced. No, it is not Potlatch 0.9a, it cannot be Potlatch is the perfect user tool and introduction to OSM. It must be API 0.6 that didn't solve all our problems, as was promised. Instead but it opened the gates of hell, a parallel universe within OSM...[/melodramatic] [start fanfare tune + brass] http://api.openstreetmap.org/api/0.6/way/8115655/full http://api.openstreetmap.org/api/0.6/node/255483811/history I probably have to blame myself; I was a fool to speed up my converter by removing the consistency checks. But since when do we support more than -180/+180 ? Looking at the data, it was modified on 2008-05-11. That was within the API 0.5 timeframe. So surely it was the 0.5 - 0.6 upgrade process that didn't catch the data anomaly, rather than the current API 0.6 code? After all, the data hasn't been changed since API 0.6 went live. It's good to see that Iván has posted a very plausible reason for why those particular longitudes were getting shown. I found the location of the lake in question on Wikipedia and see that the latitudes have changed significantly as well. I've written a little bit of code to revert the nodes to their most recent valid positions and committed the result after checking that it looked reasonable in JOSM. Gregory ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Problems with Planet-Excerpt Import into apidb
Hi Holger, These tasks haven't had a lot of testing since the 0.6 API change. It should work but I haven't tried it with large files. Are you able to replicate the problem with a much smaller dataset or does it just happen with very large imports? The unit tests within osmosis are passing but they only import about a dozen rows. Brett Holger Schöner wrote: Hi, After the code seemed to have stabilized somewhat after the api version increment, I wanted to try out replicating the OSM database locally. As I only have a laptop with 2GB main memory and about 100GB harddisk available, I expected the whole planet to be a chunk a little too large for me, and tried to import the Geofabrik Europe extract. For this I used the svn- version of Osmosis (0.31.1 is says) with the apidb06-pgsql-latest.sql db schema. Dependiing on whether I split this schema into two parts before the COPY statements, key and index creation (as I remember reading on the list some time ago) or setting up the db completely before the osmosis import, I run into different problems: If do not split up the sql-script before importing, osmosis seems to run forever, and even after 2 days, the nodes and current_nodes tables do not contain any rows. If I split the sql-script and setup the keys and indexes only after the first nodes arrive in the db, then after some time (the nodes table contains 95625500 rows and the current_nodes table contains 36456661 rows; are these all of Europe? Ways and relations are empty), osmosis terminates with the following error message: -- bzcat europe.osm.bz2 | JAVACMD_OPTIONS='-Xmx1536m' /usr/local/bin/osmosis --read-xml-0.6 file=/dev/stdin --write-apidb-0.6 populateCurrentTables=yes validateSchemaVersion=no host=localhost database=osm user=*** password= 14.05.2009 22:59:33 org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.31.1 14.05.2009 22:59:33 org.openstreetmap.osmosis.core.Osmosis run INFO: Preparing pipeline. 14.05.2009 22:59:34 org.openstreetmap.osmosis.core.Osmosis run INFO: Launching pipeline execution. 14.05.2009 22:59:34 org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline executing, waiting for completion. 15.05.2009 12:12:31 org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion SCHWERWIEGEND: Thread for task 1-read-xml-0.6 failed org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to load current node tags. at org.openstreetmap.osmosis.core.apidb.v0_6.ApidbWriter.complete(ApidbWriter.java:944) at org.openstreetmap.osmosis.core.xml.v0_6.XmlReader.run(XmlReader.java:110) at java.lang.Thread.run(Thread.java:619) Caused by: org.postgresql.util.PSQLException: FEHLER: Verklemmung (Deadlock) entdeckt Detail: Prozess 6577 wartet auf RowExclusiveLock-Sperre auf Relation 441591 der Datenbank 440675; blockiert von Prozess 12658. Prozess 12658 wartet auf AccessExclusiveLock-Sperre auf Relation 441599 der Datenbank 440675; blockiert von Prozess 6577. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1592) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1327) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:192) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:451) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:350) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:343) at org.openstreetmap.osmosis.core.apidb.v0_6.ApidbWriter.complete(ApidbWriter.java:941) ... 2 more 15.05.2009 12:12:31 org.openstreetmap.osmosis.core.Osmosis main SCHWERWIEGEND: Execution aborted. org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed. at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146) at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:85) at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:30) -- (I also tried without giving it populateCurrentTables=yes.) the postgresql log shows me at the same time: -- ... 2009-05-15 08:37:56 CEST HINWEIS: ALTER TABLE / ADD PRIMARY KEY erstellt implizit einen Index »way_tags_pkey« für Tabelle »way_tags« 2009-05-15 08:54:43 CEST HINWEIS: ALTER TABLE / ADD PRIMARY KEY erstellt implizit einen Index »ways_pkey« für Tabelle »ways« 2009-05-15 12:12:30 CEST FEHLER: Verklemmung (Deadlock) entdeckt 2009-05-15 12:12:30 CEST DETAIL: Prozess 6577 wartet auf RowExclusiveLock- Sperre auf Relation 441591 der Datenbank 440675; blockiert von Prozess 12658. Prozess 12658 wartet auf AccessExclusiveLock-Sperre auf Relation 441599 der Datenbank 440675; blockiert von
Re: [OSM-dev] Guys I trusted you, I removed my checks...
-Original Message- From: dev-boun...@openstreetmap.org [mailto:dev- boun...@openstreetmap.org] On Behalf Of Gregory Williams Sent: 19 May 2009 13:09 To: OSM-Dev Openstreetmap Subject: Re: [OSM-dev] Guys I trusted you, I removed my checks... -Original Message- From: dev-boun...@openstreetmap.org [mailto:dev- boun...@openstreetmap.org] On Behalf Of Stefan de Konink Sent: 18 May 2009 23:57 To: OSM-Dev Openstreetmap Subject: [OSM-dev] Guys I trusted you, I removed my checks... ...but again this trust was misplaced. No, it is not Potlatch 0.9a, it cannot be Potlatch is the perfect user tool and introduction to OSM. It must be API 0.6 that didn't solve all our problems, as was promised. Instead but it opened the gates of hell, a parallel universe within OSM...[/melodramatic] [start fanfare tune + brass] http://api.openstreetmap.org/api/0.6/way/8115655/full http://api.openstreetmap.org/api/0.6/node/255483811/history I probably have to blame myself; I was a fool to speed up my converter by removing the consistency checks. But since when do we support more than -180/+180 ? Looking at the data, it was modified on 2008-05-11. That was within the API 0.5 timeframe. So surely it was the 0.5 - 0.6 upgrade process that didn't catch the data anomaly, rather than the current API 0.6 code? After all, the data hasn't been changed since API 0.6 went live. It's good to see that Iván has posted a very plausible reason for why those particular longitudes were getting shown. I found the location of the lake in question on Wikipedia and see that the latitudes have changed significantly as well. I've written a little bit of code to revert the nodes to their most recent valid positions and committed the result after checking that it looked reasonable in JOSM. Ooops! I'd actually rolled it back two versions by mistake there (silly combination of looking for the previous version and a version with valid lat / lon in the code). Corrected now. Gregory ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Problems with Planet-Excerpt Import into apidb
Hi Brett, These tasks haven't had a lot of testing since the 0.6 API change. It should work but I haven't tried it with large files. Are you able to replicate the problem with a much smaller dataset or does it just happen with very large imports? The unit tests within osmosis are passing but they only import about a dozen rows. Okay, I tried the berlin.osm.bz2 from Geofabrik now (around 8MB). When first setting up the db completely, the import ran through without any errors in a few minutes. When applying the db schema in two steps, I got a deadlock again, but the import still seemed to work okay (osmosis terminated without error, and nodes, current_nodes, ways, current_ways, relations, current_relations were all filled with the same amount of rows as before): -- ... ALTER TABLE ALTER TABLE psql:/home/numenor/data/gps/osm/osmosis/apidb06-pgsql-latest-2.sql:707: FEHLER: Verklemmung (Deadlock) entdeckt DETAIL: Prozess 30441 wartet auf AccessExclusiveLock-Sperre auf Relation 443051 der Datenbank 441959; blockiert von Prozess 30240. Prozess 30240 wartet auf AccessShareLock-Sperre auf Relation 443041 der Datenbank 441959; blockiert von Prozess 30441. ALTER TABLE ALTER TABLE ... -- Could there be an error in the Europe extract at Geofabrik? I used the one from 2009-05-09 08:36 . Or should I just endure an Europe import (probably of seval days) with no splitting of the schema initialization, but after which I can stay up to date with the diffs? Holger Holger Schöner wrote: After the code seemed to have stabilized somewhat after the api version increment, I wanted to try out replicating the OSM database locally. As I only have a laptop with 2GB main memory and about 100GB harddisk available, I expected the whole planet to be a chunk a little too large for me, and tried to import the Geofabrik Europe extract. For this I used the svn- version of Osmosis (0.31.1 is says) with the apidb06-pgsql-latest.sql db schema. Dependiing on whether I split this schema into two parts before the COPY statements, key and index creation (as I remember reading on the list some time ago) or setting up the db completely before the osmosis import, I run into different problems: If do not split up the sql-script before importing, osmosis seems to run forever, and even after 2 days, the nodes and current_nodes tables do not contain any rows. If I split the sql-script and setup the keys and indexes only after the first nodes arrive in the db, then after some time (the nodes table contains 95625500 rows and the current_nodes table contains 36456661 rows; are these all of Europe? Ways and relations are empty), osmosis terminates with the following error message: -- bzcat europe.osm.bz2 | JAVACMD_OPTIONS='-Xmx1536m' /usr/local/bin/osmosis --read-xml-0.6 file=/dev/stdin --write-apidb-0.6 populateCurrentTables=yes validateSchemaVersion=no host=localhost database=osm user=*** password= 14.05.2009 22:59:33 org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.31.1 14.05.2009 22:59:33 org.openstreetmap.osmosis.core.Osmosis run INFO: Preparing pipeline. 14.05.2009 22:59:34 org.openstreetmap.osmosis.core.Osmosis run INFO: Launching pipeline execution. 14.05.2009 22:59:34 org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline executing, waiting for completion. 15.05.2009 12:12:31 org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion SCHWERWIEGEND: Thread for task 1-read-xml-0.6 failed org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to load current node tags. at org.openstreetmap.osmosis.core.apidb.v0_6.ApidbWriter.complete(ApidbWri ter.java:944) at org.openstreetmap.osmosis.core.xml.v0_6.XmlReader.run(XmlReader.java:11 0) at java.lang.Thread.run(Thread.java:619) Caused by: org.postgresql.util.PSQLException: FEHLER: Verklemmung (Deadlock) entdeckt Detail: Prozess 6577 wartet auf RowExclusiveLock-Sperre auf Relation 441591 der Datenbank 440675; blockiert von Prozess 12658. Prozess 12658 wartet auf AccessExclusiveLock-Sperre auf Relation 441599 der Datenbank 440675; blockiert von Prozess 6577. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExec utorImpl.java:1592) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorIm pl.java:1327) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java :192) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statem ent.java:451) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJd bc2Statement.java:350) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statem ent.java:343) at org.openstreetmap.osmosis.core.apidb.v0_6.ApidbWriter.complete(ApidbWri ter.java:941) ... 2 more 15.05.2009 12:12:31 org.openstreetmap.osmosis.core.Osmosis main
Re: [OSM-dev] Guys I trusted you, I removed my checks...
On Tue, May 19, 2009 at 1:08 PM, Gregory Williams gregory.willi...@purplegeodesoftware.co.uk wrote: Looking at the data, it was modified on 2008-05-11. That was within the API 0.5 timeframe. So surely it was the 0.5 - 0.6 upgrade process that didn't catch the data anomaly, rather than the current API 0.6 code? After all, the data hasn't been changed since API 0.6 went live. the migration didn't even check lat/lon - i hadn't considered the possibility that there was already duff data in the 0.5 db! richard had a very plausible explanation for how they got in though. we should definitely ban the editor which was used ;-) cheers, matt ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Guys I trusted you, I removed my checks...
Matt Amos wrote: On Tue, May 19, 2009 at 1:08 PM, Gregory Williams gregory.willi...@purplegeodesoftware.co.uk wrote: Looking at the data, it was modified on 2008-05-11. That was within the API 0.5 timeframe. So surely it was the 0.5 - 0.6 upgrade process that didn't catch the data anomaly, rather than the current API 0.6 code? After all, the data hasn't been changed since API 0.6 went live. the migration didn't even check lat/lon - i hadn't considered the possibility that there was already duff data in the 0.5 db! I did a count earlier on and there are a couple of hundred with one or both coordinates out of bounds. Some of those might be deleted though as I didn't check the visible flag. Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] merkaator debuild still fails ...
On Dienstag 19 Mai 2009, Chris Browet wrote: Solved. Do not forget to rerun qmake. Please not that there is a merkaartor-specific list at http://lists.openstreetmap.org/listinfo/merkaartor;. Dev is rather for the core openstreetmap. Could someone have a look at this please ... New log is also at: http://www.gpsdrive.de/build_cluster/merkaartor/debian-squeeze-64/debuild.log.shtml Now I'm getting: dh_installexamples Data/* cp: cannot stat `Data/*': No such file or directory dh_installexamples: command returned error code 256 Build-Cluster Overview is at: http://www.gpsdrive.de/build_cluster/results.shtml Jörg (Germany, Tettnang) http://www.ostertag.name/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[josm-dev] trac overcautios spam filter
I tried to file a new ticket in trac but it was rejected for too much activity from my IP (Spam) - which is hard to believe, as my IP is dynamically assigned and my last post was 3 days ago. I registered to solve but there might be a problem anyways, that could occur to others not registered. Martin ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
