Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Jon Burgess wrote: On Fri, 2008-11-14 at 21:08 +0200, Rahkonen Jukka wrote: 2008/11/14 Rahkonen Jukka [EMAIL PROTECTED]: This version imports Finland.osm dataset OK. There are slight differencies in the number of features imported by this and Artem's version: Old points: 24578 lines: 97223 polygons: 48305 New points: 23680 lines: 96889 polygons: 50316 I will do cross check later by selecting features imported by the old but not with the new and vice versa. -Jukka- I'd guess that its just a change in the osm2pgsql style definition, which determines which features are interesting, and whether those features should be treated as polygons or as linestrings. I'd say that the new one has problems at least with multipolygon relations. For example a multipolygon, relation with OSM_ID 4230, is imported as separate polygons, outer ring having OSM_ID 12298022. That's probably why the number of polygons is bigger, there are lots of lakes with holes in the data. I have not yet isolated any line or point differencies, I am just learning how to query them visible from PostGIS. I am sorry I can't give lat/lon coordinates now, I have reprojected the data into Finnish KKJ system in the database. I will reproject it to epsg:4326 before next investigations. Did you import the data with --slim? There is a bug in the current code which which breaks multipolygons unless the slim mode is used. I see relation 4230 appearing as a polygon with lots of holes: foo= select osm_id,name,natural,NumInteriorRings(way) from planet_osm_polygon where osm_id in (-4230,12298022); osm_id | name| natural | numinteriorrings +---+-+-- -4230 | Konnivesi | water | 67 (1 row) No, I did not use the --slim switch. As advertised, it is much slower to import with slim mode, but it does import polygons with holes correctly. Differencies in feature count was now 97223-96970 for lines and 48305-48581 for polygons. I created difference layers from tables imported with the old osm2pgsql.exe and this new one in slim mode. Query may be stupid, but I think that it selects reliably objects imported with the old one (named OSM_) which do not have a pair with same OSM_ID in the other table (named GOSM_) SQL is like this create table diff2 as (select * from (select osm.*, gosm.osm_id as goo, gosm.name as gname from osm_polygon osm LEFT JOIN gosm_polygon gosm using(osm_id)) as foo where foo.goo is null); Some findings: - Old program gave positive OSM_IDs for features created from relations while the new seems to give them negative values. Obviously the code has been changed in between. I guess and hope that this change makes OSM_ID as unique identifier so it can be used as a primary key in the imported PostGIS tables. Until now I have been forced to create a new OID field for primary key. - There are some features that are not imported at all by the new version. At least areas with tag shop=supermarket and no other tags except name (way 4881398). It does not appear on Mapnik map either, so the program works for me in the similar way than in Mapnik slippy map production. - Combination highway=[any type], area=yes comes now correctly as polygon, while they used to be lines. - Major part of differencies in number of line features are due to riverbanks and highways marked with area=yes tag. They get now correctly converted into polygons and can be found there. For me it looks like it is safe to use the new osm2pgsql.exe, but only in slim mode. -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Sat, Nov 15, 2008 at 12:06 PM, Rahkonen Jukka [EMAIL PROTECTED] wrote: Some findings: - Old program gave positive OSM_IDs for features created from relations while the new seems to give them negative values. Obviously the code has been changed in between. I guess and hope that this change makes OSM_ID as unique identifier so it can be used as a primary key in the imported PostGIS tables. Until now I have been forced to create a new OID field for primary key. Nope, because some OSM features translate to more than one postgis feature and so they get the same OSM id. - There are some features that are not imported at all by the new version. At least areas with tag shop=supermarket and no other tags except name (way 4881398). It does not appear on Mapnik map either, so the program works for me in the similar way than in Mapnik slippy map production. Is the shop tag mentioned in the style config? For me it looks like it is safe to use the new osm2pgsql.exe, but only in slim mode. Non-slim mode is basically fine as long as you don't need anything with relations, otherwise... Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Sat, 2008-11-15 at 13:06 +0200, Rahkonen Jukka wrote: Jon Burgess wrote: On Fri, 2008-11-14 at 21:08 +0200, Rahkonen Jukka wrote: I'd say that the new one has problems at least with multipolygon relations. For example a multipolygon, relation with OSM_ID 4230, is imported as separate polygons, outer ring having OSM_ID 12298022. That's probably why the number of polygons is bigger, there are lots of lakes with holes in the data. I have not yet isolated any line or point differencies, I am just learning how to query them visible from PostGIS. I am sorry I can't give lat/lon coordinates now, I have reprojected the data into Finnish KKJ system in the database. I will reproject it to epsg:4326 before next investigations. Did you import the data with --slim? There is a bug in the current code which which breaks multipolygons unless the slim mode is used. I see relation 4230 appearing as a polygon with lots of holes: foo= select osm_id,name,natural,NumInteriorRings(way) from planet_osm_polygon where osm_id in (-4230,12298022); osm_id | name| natural | numinteriorrings +---+-+-- -4230 | Konnivesi | water | 67 (1 row) No, I did not use the --slim switch. As advertised, it is much slower to import with slim mode, but it does import polygons with holes correctly. Differencies in feature count was now 97223-96970 for lines and 48305-48581 for polygons. I created difference layers from tables imported with the old osm2pgsql.exe and this new one in slim mode. Query may be stupid, but I think that it selects reliably objects imported with the old one (named OSM_) which do not have a pair with same OSM_ID in the other table (named GOSM_) SQL is like this create table diff2 as (select * from (select osm.*, gosm.osm_id as goo, gosm.name as gname from osm_polygon osm LEFT JOIN gosm_polygon gosm using(osm_id)) as foo where foo.goo is null); Some findings: - Old program gave positive OSM_IDs for features created from relations while the new seems to give them negative values. Obviously the code has been changed in between. I guess and hope that this change makes OSM_ID as unique identifier so it can be used as a primary key in the imported PostGIS tables. Until now I have been forced to create a new OID field for primary key. When we create nodes for parking areas the way ID is used to identify the new node which could cause a clash. We could make this a negative ID which would help. I think there is an edge case in the multipolygon support which can cause multiple linear ways to be created with the same relation ID if the rings are not closed. In short, I don't think we can give any guarantees about the uniqueness of the osm_id column. - There are some features that are not imported at all by the new version. At least areas with tag shop=supermarket and no other tags except name (way 4881398). It does not appear on Mapnik map either, so the program works for me in the similar way than in Mapnik slippy map production. I don't think the 'shop' tag has ever been included in the list of imported tags. It is definitely not being rendered by the current osm.xml file. - Combination highway=[any type], area=yes comes now correctly as polygon, while they used to be lines. - Major part of differencies in number of line features are due to riverbanks and highways marked with area=yes tag. They get now correctly converted into polygons and can be found there. This is one of several new features introduced since the last time the Windows version was built. Another big feature is the support for diff imports. For me it looks like it is safe to use the new osm2pgsql.exe, but only in slim mode. Thanks for doing the testing. Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Jon Burgess wrote: On Sat, 2008-11-15 at 13:06 +0200, Rahkonen Jukka wrote: In short, I don't think we can give any guarantees about the uniqueness of the osm_id column. Good to know. I am playing with Finnish and Scandinavian data only and I set keep PostGIS using WITH_OIDS as default and thus I get primary keys automatically. Not really a problem for me, but those who are playing with more data and perhaps have a remote database without db-admin rights need to create the primary key field themselves. Primary key is compulsory for many uses. But knowing the situation helps. - There are some features that are not imported at all by the new version. At least areas with tag shop=supermarket and no other tags except name (way 4881398). It does not appear on Mapnik map either, so the program works for me in the similar way than in Mapnik slippy map production. I don't think the 'shop' tag has ever been included in the list of imported tags. It is definitely not being rendered by the current osm.xml file. Right, did not remember that shop wan one of those extra tags that was added to osm2pgsql.exe. It is possible to read map features page so that giving shop=supermarket as an area is supported, but not all shop=something. Osmarender seems to render it. For on-demand rendering it would be more simple to keep all shops as points and perhaps colour the building according to building=shop or building=commercial or the like. But I have understood that now I can just edit the default.style file for having some own tags, for example max_speed if I'd like to colour highways according to that. - Combination highway=[any type], area=yes comes now correctly as polygon, while they used to be lines. - Major part of differencies in number of line features are due to riverbanks and highways marked with area=yes tag. They get now correctly converted into polygons and can be found there. This is one of several new features introduced since the last time the Windows version was built. Another big feature is the support for diff imports. For me it looks like it is safe to use the new osm2pgsql.exe, but only in slim mode. Thanks for doing the testing. Thanks for compiling. By the way, the final report is missing now, that used to look like this Node stats: total(2658737), max(312060206) Way stats: total(175231), max(28408381) Relation stats: total(836), max(51578) -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Sat, 2008-11-15 at 14:25 +0200, Rahkonen Jukka wrote: Jon Burgess wrote: On Sat, 2008-11-15 at 13:06 +0200, Rahkonen Jukka wrote: In short, I don't think we can give any guarantees about the uniqueness of the osm_id column. Good to know. I am playing with Finnish and Scandinavian data only and I set keep PostGIS using WITH_OIDS as default and thus I get primary keys automatically. Not really a problem for me, but those who are playing with more data and perhaps have a remote database without db-admin rights need to create the primary key field themselves. Primary key is compulsory for many uses. But knowing the situation helps. - There are some features that are not imported at all by the new version. At least areas with tag shop=supermarket and no other tags except name (way 4881398). It does not appear on Mapnik map either, so the program works for me in the similar way than in Mapnik slippy map production. I don't think the 'shop' tag has ever been included in the list of imported tags. It is definitely not being rendered by the current osm.xml file. Right, did not remember that shop wan one of those extra tags that was added to osm2pgsql.exe. It is possible to read map features page so that giving shop=supermarket as an area is supported, but not all shop=something. Osmarender seems to render it. For on-demand rendering it would be more simple to keep all shops as points and perhaps colour the building according to building=shop or building=commercial or the like. But I have understood that now I can just edit the default.style file for having some own tags, for example max_speed if I'd like to colour highways according to that. Mapnik does not render everything listed in Map_Features but building=* will render. The default.style will let you customise this quite easily which is a nice new feature for the Windows version. Good luck with max_speed. I believe there are lots of variations in the way the data has been entered which will probably cause issues. - Combination highway=[any type], area=yes comes now correctly as polygon, while they used to be lines. - Major part of differencies in number of line features are due to riverbanks and highways marked with area=yes tag. They get now correctly converted into polygons and can be found there. This is one of several new features introduced since the last time the Windows version was built. Another big feature is the support for diff imports. For me it looks like it is safe to use the new osm2pgsql.exe, but only in slim mode. Thanks for doing the testing. Thanks for compiling. By the way, the final report is missing now, that used to look like this Node stats: total(2658737), max(312060206) Way stats: total(175231), max(28408381) Relation stats: total(836), max(51578) I see these in the output when I run the program as below: $ ./osm2pgsql.exe -W --slim -U foo -d foo cyprus.osm.bz2 osm2pgsql SVN version 0.55-20081113 $Rev: 10464 $ Password:foo Using projection SRS 900913 (Spherical Mercator) Setting up table: planet_osm_point Setting up table: planet_osm_line Setting up table: planet_osm_polygon Setting up table: planet_osm_roads Mid: pgsql, scale=100, cache=800MB, maxblocks=102401*zd Setting up table: planet_osm_nodes *** WARNING: intarray contrib module not installed *** The resulting database will not be usable for applying diffs. NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index planet_osm_nodes_pkey for table planet_osm_nodes Setting up table: planet_osm_ways NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index planet_osm_ways_pkey for table planet_osm_ways Setting up table: planet_osm_rels NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index planet_osm_rels_pkey for table planet_osm_rels Reading in file: cyprus.osm.bz2 Processing: Node(213k) Way(20k) Relation(0k) Node stats: total(213472), max(312053827) Way stats: total(20617), max(28408030) Relation stats: total(3), max(48708) Going over pending ways processing way (1k) Going over pending relations node cache: stored: 213472(100.00%), storage efficiency: 2.78%, hit rate: 100.00% Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
2008/11/14 Rahkonen Jukka [EMAIL PROTECTED]: This version imports Finland.osm dataset OK. There are slight differencies in the number of features imported by this and Artem's version: Old points: 24578 lines: 97223 polygons: 48305 New points: 23680 lines: 96889 polygons: 50316 I will do cross check later by selecting features imported by the old but not with the new and vice versa. -Jukka- I'd guess that its just a change in the osm2pgsql style definition, which determines which features are interesting, and whether those features should be treated as polygons or as linestrings. -- Regards, Thomas Wood (Edgemaster) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Lähettäjä: Thomas Wood [mailto:[EMAIL PROTECTED] 2008/11/14 Rahkonen Jukka [EMAIL PROTECTED]: This version imports Finland.osm dataset OK. There are slight differencies in the number of features imported by this and Artem's version: Old points: 24578 lines: 97223 polygons: 48305 New points: 23680 lines: 96889 polygons: 50316 I will do cross check later by selecting features imported by the old but not with the new and vice versa. -Jukka- I'd guess that its just a change in the osm2pgsql style definition, which determines which features are interesting, and whether those features should be treated as polygons or as linestrings. I'd say that the new one has problems at least with multipolygon relations. For example a multipolygon, relation with OSM_ID 4230, is imported as separate polygons, outer ring having OSM_ID 12298022. That's probably why the number of polygons is bigger, there are lots of lakes with holes in the data. I have not yet isolated any line or point differencies, I am just learning how to query them visible from PostGIS. I am sorry I can't give lat/lon coordinates now, I have reprojected the data into Finnish KKJ system in the database. I will reproject it to epsg:4326 before next investigations. -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Fri, 2008-11-14 at 21:08 +0200, Rahkonen Jukka wrote: Lähettäjä: Thomas Wood [mailto:[EMAIL PROTECTED] 2008/11/14 Rahkonen Jukka [EMAIL PROTECTED]: This version imports Finland.osm dataset OK. There are slight differencies in the number of features imported by this and Artem's version: Old points: 24578 lines: 97223 polygons: 48305 New points: 23680 lines: 96889 polygons: 50316 I will do cross check later by selecting features imported by the old but not with the new and vice versa. -Jukka- I'd guess that its just a change in the osm2pgsql style definition, which determines which features are interesting, and whether those features should be treated as polygons or as linestrings. I'd say that the new one has problems at least with multipolygon relations. For example a multipolygon, relation with OSM_ID 4230, is imported as separate polygons, outer ring having OSM_ID 12298022. That's probably why the number of polygons is bigger, there are lots of lakes with holes in the data. I have not yet isolated any line or point differencies, I am just learning how to query them visible from PostGIS. I am sorry I can't give lat/lon coordinates now, I have reprojected the data into Finnish KKJ system in the database. I will reproject it to epsg:4326 before next investigations. Did you import the data with --slim? There is a bug in the current code which which breaks multipolygons unless the slim mode is used. I see relation 4230 appearing as a polygon with lots of holes: foo= select osm_id,name,natural,NumInteriorRings(way) from planet_osm_polygon where osm_id in (-4230,12298022); osm_id | name| natural | numinteriorrings +---+-+-- -4230 | Konnivesi | water | 67 (1 row) Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Thu, Nov 13, 2008 at 7:24 AM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I made a quick test with the program and yesterdays Finland.osm.bz2 from Geofabrik.de. Program starts OK and reading in data is perhaps faster than with the older osm2pgsql.exe. However, import was not successful because of some error in the data. Message is this: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.68304e+006 8.95213e+006, 2.68415e+006 8.95259e+006 and 2.68415e+006 8.95259e+006, 2.68391e+ 006 8.95307e+006 2.68415e+006 8.95259e+006 That's, uh, wierd. Never seen that message before. We do use geos but I didn't think we did anything to trigger this... There's no other helpful information? Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Martijn van Oosterhout wrote: On Thu, Nov 13, 2008 at 7:24 AM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I made a quick test with the program and yesterdays Finland.osm.bz2 from Geofabrik.de. Program starts OK and reading in data is perhaps faster than with the older osm2pgsql.exe. However, import was not successful because of some error in the data. Message is this: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.68304e+006 8.95213e+006, 2.68415e+006 8.95259e+006 and 2.68415e+006 8.95259e+006, 2.68391e+ 006 8.95307e+006 2.68415e+006 8.95259e+006 That's, uh, wierd. Never seen that message before. We do use geos but I didn't think we did anything to trigger this... There's no other helpful information? No, unfortunately not. I made two more tests with another computer. 1. Test data: Isle of Wight data from Kosmos v. 2.2.25.2 sample data. Result: OK 2. Test data: Finland.osm.bz2 from 21st October 2008. Result: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.34623e+006 1.07286e+007, 2.34611e+006 1.07287e+007 and 2.34593e+006 1.07281e+007, 2.34684e+ 006 1.07301e+007 2.34618e+006 1.07286e+007 For comparison the same data got imported with the Artem's version of osm2pgsql, even that too gave an odd error. This error is from different place and obviously for some other reason: ElevationMatrix::add(2.8979e+006 8.63408e+006 1.7e-308): Coordinate does not overlap grid extent: IllegalArgumentException: ElevationMatrix::getCell got a Coord inate out of grid extent (Env[2.89876e+006:2.90052e+006,8.6307e+006:8.63318e+006 ]) - cols:3 rows:3 Conclusion: Jon's osm2pgsql.exe works, but something in the system is perhaps too sensible for OSM data errors. If error is something Windows related could be verified if somebody who has osm2pgsql on a Linux box could have a try with Finnish data from Geofabrik (http://download.geofabrik.de/osm/europe/finland.osm.bz2 http://download.geofabrik.de/osm/europe/finland.osm.bz2 )? It takes about five minutes to run. -Jukka- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Thu, Nov 13, 2008 at 3:03 PM, Rahkonen Jukka [EMAIL PROTECTED] wrote: Conclusion: Jon's osm2pgsql.exe works, but something in the system is perhaps too sensible for OSM data errors. If error is something Windows related could be verified if somebody who has osm2pgsql on a Linux box could have a try with Finnish data from Geofabrik (http://download.geofabrik.de/osm/europe/finland.osm.bz2 http://download.geofabrik.de/osm/europe/finland.osm.bz2 )? It takes about five minutes to run. Just tried it here and it works fine (Linux). Mist be something windows specific. Maybe someone put in an asserts build or something? Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Hello, I'm trying it on Windows XP with access to a Postgis database and I get this output: === C:\downloads\osm2pgsql\osm2pgsqlosm2pgsql --database geo --latlong --username postgres --password postgres --host noruega --port 5432 --verbose ./planet.osm osm2pgsql SVN version 0.55-20081112 $Rev: 10464 $ Password: Using projection SRS 4326 (Latlong) Setting up table: planet_osm_point Setting up table: planet_osm_line Setting up table: planet_osm_polygon Setting up table: planet_osm_roads Mid: Ram, scale=1000 Reading in file: postgres error while opening file postgres C:\downloads\osm2pgsql\osm2pgsql === Why does it ask for a password again? cheers, Lucas De: [EMAIL PROTECTED] en nombre de Martijn van Oosterhout Enviado el: jue 13/11/2008 16:03 Para: Rahkonen Jukka CC: talk@openstreetmap.org Asunto: Re: [OSM-talk] Wanted: Osm2pgsql.exe developer On Thu, Nov 13, 2008 at 3:03 PM, Rahkonen Jukka [EMAIL PROTECTED] wrote: Conclusion: Jon's osm2pgsql.exe works, but something in the system is perhaps too sensible for OSM data errors. If error is something Windows related could be verified if somebody who has osm2pgsql on a Linux box could have a try with Finnish data from Geofabrik (http://download.geofabrik.de/osm/europe/finland.osm.bz2 http://download.geofabrik.de/osm/europe/finland.osm.bz2 )? It takes about five minutes to run. Just tried it here and it works fine (Linux). Mist be something windows specific. Maybe someone put in an asserts build or something? Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
I got same result on WinXP Error see: http://dev.openstreetmap.org/~steve8/errorosm2pgsql.png STEVE -Original Message- From: [EMAIL PROTECTED] on behalf of Jukka Rahkonen Sent: Thu 11/13/2008 6:24 AM To: talk@openstreetmap.org Cc: Subject: Re: [OSM-talk] Wanted: Osm2pgsql.exe developer Jon Burgess jburgess777 at googlemail.com writes: I made a successful attempt at cross compiling osm2pgsql.exe using MinGW on Linux. The executable seems to work OK under Wine. Let me know if it works on a native Windows machine: http://tile.openstreetmap.org/direct/osm2pgsql.zip I made a quick test with the program and yesterdays Finland.osm.bz2 from Geofabrik.de. Program starts OK and reading in data is perhaps faster than with the older osm2pgsql.exe. However, import was not successful because of some error in the data. Message is this: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.68304e+006 8.95213e+006, 2.68415e+006 8.95259e+006 and 2.68415e+006 8.95259e+006, 2.68391e+ 006 8.95307e+006 2.68415e+006 8.95259e+006 This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. So obviously nothing wrong with compiling. I wonder if complete stop at topology error is necessary. In this case it makes importing the data impossible before the error is corrected. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Sorry, I had not read the help. It works fine now. Lucas De: [EMAIL PROTECTED] en nombre de Juan Lucas Dominguez Rubio Enviado el: jue 13/11/2008 17:08 Para: Martijn van Oosterhout; Rahkonen Jukka CC: talk@openstreetmap.org Asunto: Re: [OSM-talk] Wanted: Osm2pgsql.exe developer Hello, I'm trying it on Windows XP with access to a Postgis database and I get this output: === C:\downloads\osm2pgsql\osm2pgsqlosm2pgsql --database geo --latlong --username postgres --password postgres --host noruega --port 5432 --verbose ./planet.osm osm2pgsql SVN version 0.55-20081112 $Rev: 10464 $ Password: Using projection SRS 4326 (Latlong) Setting up table: planet_osm_point Setting up table: planet_osm_line Setting up table: planet_osm_polygon Setting up table: planet_osm_roads Mid: Ram, scale=1000 Reading in file: postgres error while opening file postgres C:\downloads\osm2pgsql\osm2pgsql === Why does it ask for a password again? cheers, Lucas De: [EMAIL PROTECTED] en nombre de Martijn van Oosterhout Enviado el: jue 13/11/2008 16:03 Para: Rahkonen Jukka CC: talk@openstreetmap.org Asunto: Re: [OSM-talk] Wanted: Osm2pgsql.exe developer On Thu, Nov 13, 2008 at 3:03 PM, Rahkonen Jukka [EMAIL PROTECTED] wrote: Conclusion: Jon's osm2pgsql.exe works, but something in the system is perhaps too sensible for OSM data errors. If error is something Windows related could be verified if somebody who has osm2pgsql on a Linux box could have a try with Finnish data from Geofabrik (http://download.geofabrik.de/osm/europe/finland.osm.bz2 http://download.geofabrik.de/osm/europe/finland.osm.bz2 )? It takes about five minutes to run. Just tried it here and it works fine (Linux). Mist be something windows specific. Maybe someone put in an asserts build or something? Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Thu, 2008-11-13 at 16:03 +0200, Rahkonen Jukka wrote: Martijn van Oosterhout wrote: On Thu, Nov 13, 2008 at 7:24 AM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I made a quick test with the program and yesterdays Finland.osm.bz2 from Geofabrik.de. Program starts OK and reading in data is perhaps faster than with the older osm2pgsql.exe. However, import was not successful because of some error in the data. Message is this: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.68304e+006 8.95213e+006, 2.68415e+006 8.95259e+006 and 2.68415e+006 8.95259e+006, 2.68391e+ 006 8.95307e+006 2.68415e+006 8.95259e+006 That's, uh, wierd. Never seen that message before. We do use geos but I didn't think we did anything to trigger this... There's no other helpful information? No, unfortunately not. I made two more tests with another computer. 1. Test data: Isle of Wight data from Kosmos v. 2.2.25.2 sample data. Result: OK 2. Test data: Finland.osm.bz2 from 21st October 2008. Result: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.34623e+006 1.07286e+007, 2.34611e+006 1.07287e+007 and 2.34593e+006 1.07281e+007, 2.34684e+ 006 1.07301e+007 2.34618e+006 1.07286e+007 I've found the source of the problem. It seems there is an issue catching the exceptions generated in the geos DLL by main program. For comparison the same data got imported with the Artem's version of osm2pgsql, even that too gave an odd error. This error is from different place and obviously for some other reason: ElevationMatrix::add(2.8979e+006 8.63408e+006 1.7e-308): Coordinate does not overlap grid extent: IllegalArgumentException: ElevationMatrix::getCell got a Coord inate out of grid extent (Env[2.89876e+006:2.90052e+006,8.6307e +006:8.63318e+006 ]) - cols:3 rows:3 Conclusion: Jon's osm2pgsql.exe works, but something in the system is perhaps too sensible for OSM data errors. If error is something Windows related could be verified if somebody who has osm2pgsql on a Linux box could have a try with Finnish data from Geofabrik (http://download.geofabrik.de/osm/europe/finland.osm.bz2 http://download.geofabrik.de/osm/europe/finland.osm.bz2 )? It takes about five minutes to run. The exceptions are often thrown when the osm2pgsql code calls geos to work out an interior point for the 'P' symbol of a parking area. If the area is bad, e.g. self-intersecting, then you'll get an exception thrown. The osm2pgsql code has a try/catch block to ignore these errors but this was failing. It tried some combinations of '-shared-libgcc' as mentioned on some of the sites below [1][2][3] but only statically linking the geos code into osm2pgsql seemed to fix it. There is an updated executable at: http://tile.openstreetmap.org/direct/osm2pgsql.zip Jon [1] http://sourceforge.net/tracker/?func=detailatid=102435aid=1790974group_id=2435 [2] https://bugs.launchpad.net/debian/+source/mingw32/+bug/195407 [3] https://bugs.launchpad.net/mingw/+bug/173116 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Jon Burgess wrote: On Thu, 2008-11-13 at 16:03 +0200, Rahkonen Jukka wrote: No, unfortunately not. I made two more tests with another computer. 1. Test data: Isle of Wight data from Kosmos v. 2.2.25.2 sample data. Result: OK 2. Test data: Finland.osm.bz2 from 21st October 2008. Result: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.34623e+006 1.07286e+007, 2.34611e+006 1.07287e+007 and 2.34593e+006 1.07281e+007, 2.34684e+ 006 1.07301e+007 2.34618e+006 1.07286e+007 I've found the source of the problem. It seems there is an issue catching the exceptions generated in the geos DLL by main program. ... The exceptions are often thrown when the osm2pgsql code calls geos to work out an interior point for the 'P' symbol of a parking area. If the area is bad, e.g. self-intersecting, then you'll get an exception thrown. The osm2pgsql code has a try/catch block to ignore these errors but this was failing. It tried some combinations of '-shared-libgcc' as mentioned on some of the sites below [1][2][3] but only statically linking the geos code into osm2pgsql seemed to fix it. There is an updated executable at: http://tile.openstreetmap.org/direct/osm2pgsql.zip This version imports Finland.osm dataset OK. There are slight differencies in the number of features imported by this and Artem's version: Old points: 24578 lines: 97223 polygons: 48305 New points: 23680 lines: 96889 polygons: 50316 I will do cross check later by selecting features imported by the old but not with the new and vice versa. -Jukka- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Wed, 2008-11-12 20:38:15 +0100, Martijn van Oosterhout [EMAIL PROTECTED] wrote: On Tue, Nov 11, 2008 at 9:54 AM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I would like to make clear that I do not demand any special services for the Windows users. I understand that this project is running on Linux and that osm2pgsql is done for Mapnik rendering and it is not meant to be used as a general OSM data conversion tool. For that purpose having an ogr2ogr driver would be a better solution. Osm2pgsql.exe is just damn fast in importing OSM data into PostGIS in Windows environment and therefore I am, if not so very willing, but anyway ready to pay for the kind developer who would add those two missing features. I have in the past asked for anyone who got osm2pgsql to compile under windows to send the patches needed to make it work. However, there seems to be a lack of windows developers out there :( I think I already built something that used PostgreSQL with Windows and the Cygwin stuff. I do have one Windows host (even with Cygwin installed) accessible at work. Maybe I can give it a try the other day. However, I don't really know how to distribute the binary. Should the needed libs be shipped along with it? Or ask the users to install enough Cygwin stuff until all library dependencies are fulfilled? There has to be someone out there which a windows devel environment would can spend an afternoon fixing any issues I don't think it'll be too hard. I'll try to give it a try. MfG, JBG -- Jan-Benedict Glaw [EMAIL PROTECTED] +49-172-7608481 Signature of: Zensur im Internet? Nein danke! the second : signature.asc Description: Digital signature ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Tue, Nov 11, 2008 at 9:54 AM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I would like to make clear that I do not demand any special services for the Windows users. I understand that this project is running on Linux and that osm2pgsql is done for Mapnik rendering and it is not meant to be used as a general OSM data conversion tool. For that purpose having an ogr2ogr driver would be a better solution. Osm2pgsql.exe is just damn fast in importing OSM data into PostGIS in Windows environment and therefore I am, if not so very willing, but anyway ready to pay for the kind developer who would add those two missing features. I have in the past asked for anyone who got osm2pgsql to compile under windows to send the patches needed to make it work. However, there seems to be a lack of windows developers out there :( It's quite probable that over time some gcc specific features have snuck in, but I can't tell what will work on windows. There has to be someone out there which a windows devel environment would can spend an afternoon fixing any issues Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
* On Wed, 2008-11-12 at 20:48 +0100, Jan-Benedict Glaw wrote: On Wed, 2008-11-12 20:38:15 +0100, Martijn van Oosterhout [EMAIL PROTECTED] wrote: On Tue, Nov 11, 2008 at 9:54 AM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I would like to make clear that I do not demand any special services for the Windows users. I understand that this project is running on Linux and that osm2pgsql is done for Mapnik rendering and it is not meant to be used as a general OSM data conversion tool. For that purpose having an ogr2ogr driver would be a better solution. Osm2pgsql.exe is just damn fast in importing OSM data into PostGIS in Windows environment and therefore I am, if not so very willing, but anyway ready to pay for the kind developer who would add those two missing features. I have in the past asked for anyone who got osm2pgsql to compile under windows to send the patches needed to make it work. However, there seems to be a lack of windows developers out there :( I think I already built something that used PostgreSQL with Windows and the Cygwin stuff. I do have one Windows host (even with Cygwin installed) accessible at work. Maybe I can give it a try the other day. I made a successful attempt at cross compiling osm2pgsql.exe using MinGW on Linux. The executable seems to work OK under Wine. Let me know if it works on a native Windows machine: http://tile.openstreetmap.org/direct/osm2pgsql.zip Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Jon Burgess jburgess777 at googlemail.com writes: I made a successful attempt at cross compiling osm2pgsql.exe using MinGW on Linux. The executable seems to work OK under Wine. Let me know if it works on a native Windows machine: http://tile.openstreetmap.org/direct/osm2pgsql.zip I made a quick test with the program and yesterdays Finland.osm.bz2 from Geofabrik.de. Program starts OK and reading in data is perhaps faster than with the older osm2pgsql.exe. However, import was not successful because of some error in the data. Message is this: terminate called after throwing an instance of 'geos::util::TopologyException' what(): TopologyException: found non-noded intersection between 2.68304e+006 8.95213e+006, 2.68415e+006 8.95259e+006 and 2.68415e+006 8.95259e+006, 2.68391e+ 006 8.95307e+006 2.68415e+006 8.95259e+006 This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. So obviously nothing wrong with compiling. I wonder if complete stop at topology error is necessary. In this case it makes importing the data impossible before the error is corrected. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Tom Hughes tom at compton.nu writes: Erik Johansson wrote: On Mon, Nov 10, 2008 at 7:14 PM, Jukka Rahkonen jukka.rahkonen at mmmtike.fi wrote: I would like to find somebody who believes he/she could make a new version of osm2pgsql Windows binaries. - user should be able to select the tags to be imported There is a file called default.style in the current version of osm2pgsql that allows you to do that. The other two I'm not sure about, API 0.6 should only add uid=## changeset=### to all objects.and osm2pgsql should handle that. Jukka's message was a bit confused - he knows (judging by the trac ticket he filed) that the current svn code supports the features he asked about. What he really wants is just somebody to compile the current code for windows. Hi, That's right, I have read the notes from svn and judged by that tags can be selected now, as well as the current code should handle PostGIS connection parameters. Last time it was not exectly the svn code that was used for compiling. The list of tags to be imported was prolonged by shop, is_in, historic, int_ref, postal_code, telephone, address and URL. In addition I remember somehow that some changes had to be done to the code before it started to work on Windows. I do not know exactly because I just received the executable, tested and accepted it but the work that was done there behind remained undocumented as far as I know. That's one reason why I think that perhaps OpenStreetMap trac could contain also current osm2pgsql Windows binaries or at least instructions about how to make them. PostGIS and osm2pgsql.exe are fine combination even for basic Windows users who are willing to start using OSM data with all the details in a simple way. Just a few steps are needed: - Install PostgreSQL and PostGIS extension with installer - Donload countywise OSM data from Geofabrik.de - Acquire osm2pgsql.exe and run it - Perhaps install missing Microsoft Visual C++ 2005 runtime components and run osm2pgsql.exe again - Learn that you must create PostgreSQL user having the same name than the logged-in Windows user, switch off the password checking run osm2pgsql.exe for the third time. Only discomfort at the moment is that PostGIS connection is always using the default parameters and no passwork can be set. Despite that everything goes very fast and reliably and I have been very satisfied with the utility. Until last week when I learned to create large georeferenced Mapnik maps in UTM projection with a nice Nik2img.py utility. I started to play by combining this Mapnik raster map, other raster maps from various WMS servers and local imagery as background layers with vector overlays coming from my WFS server. This is rather a nice and fast way to visualise what ever features there exists in the database without a need to make the whole load of rendering rules needed for making a nice map. And because overlays are vectors I can have all the attribute data available as well. I can already visualise cycleways, footways, streets without a name, buildings according to their tags etc, but I cannot colour highways according to max_speed tags or pharmacies according to opening hours, or cycleways according to the surface tag before I can load a bit more tags into PostGIS. I would like to make clear that I do not demand any special services for the Windows users. I understand that this project is running on Linux and that osm2pgsql is done for Mapnik rendering and it is not meant to be used as a general OSM data conversion tool. For that purpose having an ogr2ogr driver would be a better solution. Osm2pgsql.exe is just damn fast in importing OSM data into PostGIS in Windows environment and therefore I am, if not so very willing, but anyway ready to pay for the kind developer who would add those two missing features. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Wanted: Osm2pgsql.exe developer
Hi, I would like to find somebody who believes he/she could make a new version of osm2pgsql Windows binaries. New version should have a couple of addiotional features compared to the existing one at http://artem.dev.openstreetmap.org/files/ - support for giving PostGIS hostname, port, username and password - user should be able to select the tags to be imported - I guess it should support API 0.6 if it is to be used in the future, or? -Jukka Rahkonen- jukka.rahkonen at mmmtike.fi ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
On Mon, Nov 10, 2008 at 7:14 PM, Jukka Rahkonen [EMAIL PROTECTED] wrote: Hi, I would like to find somebody who believes he/she could make a new version of osm2pgsql Windows binaries. - user should be able to select the tags to be imported There is a file called default.style in the current version of osm2pgsql that allows you to do that. The other two I'm not sure about, API 0.6 should only add uid=## changeset=### to all objects.and osm2pgsql should handle that. Sorry I can't help with cross compile to Windows. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wanted: Osm2pgsql.exe developer
Erik Johansson wrote: On Mon, Nov 10, 2008 at 7:14 PM, Jukka Rahkonen [EMAIL PROTECTED] wrote: I would like to find somebody who believes he/she could make a new version of osm2pgsql Windows binaries. - user should be able to select the tags to be imported There is a file called default.style in the current version of osm2pgsql that allows you to do that. The other two I'm not sure about, API 0.6 should only add uid=## changeset=### to all objects.and osm2pgsql should handle that. Jukka's message was a bit confused - he knows (judging by the trac ticket he filed) that the current svn code supports the features he asked about. What he really wants is just somebody to compile the current code for windows. Tom -- Tom Hughes ([EMAIL PROTECTED]) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
