[OSM-dev] good way to do an automated import
Hello, I started writing the tool to do import of TMC-LocationCodeLists into OSM (will be used for Germany at first but there are lots of other countries with TMC-data so I try to make it generic and user-friendly). One very fundamental question has come up what is the best or prefered way to have the tool output it's changes? * It can download the current state of the changed entities and write an osmChange (full support of the changeset-comment+generator) http://wiki.openstreetmap.org/wiki/OsmChange * It can download the current state of the changed entities and write a josm-file (could be easy to review) http://wiki.openstreetmap.org/wiki/JOSM_file_format * It can upload it's changes directly to the API (no review) I would like to be able to run it, then review the changes it wants to make in an editor and only apply them if a large enough sample looks fine. I also need to deal with users changing entities between the time I download them and I upload my changes. Should I make this one large changeset or one changeset per type (roads, segments, points) or one changeset per imported entity (e.g. a motorway, some segments and a number of points)? I will have my tool store localy what elements have already been imported and would like it to store the IDs assigned to them by the API0.6 and the changeset-id (in case I need to rollback something later). How could I do that? Marcus ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] good way to do an automated import
Hi, marcus.wolsc...@googlemail.com wrote: I will have my tool store localy what elements have already been imported and would like it to store the IDs assigned to them by the API0.6 and the changeset-id (in case I need to rollback something later). How could I do that? If you do a diff upload against 0.6 then you get a special XML document in return that maps your negatvie Ids to those assigned by the API. JOSM has a class named DiffResultReader or so that parses this response if you want an example. Bye Frederik ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] good way to do an automated import
2009/4/8 marcus.wolsc...@googlemail.com: Hello, I started writing the tool to do import of TMC-LocationCodeLists into OSM (will be used for Germany at first but there are lots of other countries with TMC-data so I try to make it generic and user-friendly). One very fundamental question has come up what is the best or prefered way to have the tool output it's changes? * It can download the current state of the changed entities and write an osmChange (full support of the changeset-comment+generator) http://wiki.openstreetmap.org/wiki/OsmChange * It can download the current state of the changed entities and write a josm-file (could be easy to review) http://wiki.openstreetmap.org/wiki/JOSM_file_format For the UK NaPTAN import, I've used a mixture of these two. I've used the JOSM format for any initial bulk imports using the perl script. I've also tweaked a few nodes which I messed up on the original import of a region using the osmChange format to modify the nodes that had been uploaded in error. * It can upload it's changes directly to the API (no review) I would like to be able to run it, then review the changes it wants to make in an editor and only apply them if a large enough sample looks fine. I also need to deal with users changing entities between the time I download them and I upload my changes. Should I make this one large changeset or one changeset per type (roads, segments, points) or one changeset per imported entity (e.g. a motorway, some segments and a number of points)? This is something I've yet to tackle, but will have to in order to merge fields I missed from the original import by accident. I will have my tool store localy what elements have already been imported and would like it to store the IDs assigned to them by the API0.6 and the changeset-id (in case I need to rollback something later). How could I do that? I use the bulk_upload.pl script, it stores the original and new ids of files in its own file format, the dumper tool can output this list to a Old: New style output which I then parsed to modify the data as required. Marcus ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- Regards, Thomas Wood (Edgemaster) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] ANNOUNCE: OpenStreetMap maps will be added to Wikimedia projects
2009/4/5 Frederik Ramm frede...@remote.org Stephan Plepelits wrote: forking lite.. :-) Something like the OpenStreetBrowser? - http://www.openstreetbrowser.org You don't need a database fork for this, you just have to integrate the data (which is happening since the OSM was born). I think we're talking about stuff that OSM has no place for (yet, perhaps). For example an article illustrating some historic displacement of people from one region to others; I don't think anyone is saying that those trans-European arrows we known from history class should be modeled as relations in OSM, to be rendered by the appropriate Mapnik rule ;-) Things like historic displacements could be implemented as a KML overlay. The SlippyMap extension already supports KML, see http://geo.jeluf.mormo.org/index.php/Main_Page Regards, JeLuF ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] [Fwd: Your new extension has been approved]
The osmMap 0.1 module for Joomla! has been approved by the Joomla extensions team. It can be downloaded at: http://extensions.joomla.org/extensions/photos--images/maps/7913/details The source code is available at: http://trac.openstreetmap.org/browser/extensions/joomla/mod_osmMap if you are interested, please help enhancing this extension. kind regards, Milo van der Linden. Original Message Subject:Your new extension has been approved Date: Wed, 8 Apr 2009 01:17:21 -0500 From: Joomla! Extensions Directory t...@extensions.joomla.org To: m...@opengeo.nl Your new extension named osmMap 0.1 module has been approved ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Server 0.6 doesn't answer ...
Hi, sending this request: PUT /api/0.6/changeset/create HTTP/1.1 Authorization: Basic x User-Agent: osm2go-libcurl/0.6.14 Host: api06.dev.openstreetmap.org Accept: */* Content-Length: 163 ?xml version=1.0 encoding=UTF-8? osm changeset tag k=created_by v=osm2go v0.6.14/ tag k=comment v=Kommentar/ /changeset /osm i either get an internal server error 500 or no reply at all. What is wrong with this request (despite the Auth i xxx'ed out)? Till ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] SoC Project: Preprocessor to add altitude information to OSM data
Hello! I have applied for SoC to write an preprocessor that adds altitude data to ways to allow better bicycle routing, etc. The complete application text is available at [1] for those of you who don't have access to Google's mentor interface. First some information about me: I study physics at the University of Regensburg [2] and I'm in the 6th term. I already took part three times in SoC and completed all projects. My hobbies are computers, electronics, cycling and swimming. I could not discuss this proposal earlier, because I was very busy moving to a new house. Now that it's over I'll have more time. I got some questions about my application by Graham Jones grahamjones...@googlemail.com and I'll discuss then here: (Quotes are copied from his mail with permission, my replies are what I wrote him with some small corrections/additions) The first is what to do with the data - do you envisage it going back into the OSM database, or straight to the routing programme? There is a debate to be had about where to put this data - some may not wish to 'clutter' the OSM database with it. The data is not meant to go into the osm database, as it would result in incorrect data soon, as the users are free to move ways around. What I can imagine is that somebody processes the planetdump each week and publishes this dump. Something similar is done with routable garmin maps. This enables everybody to use the data without having to download all SRTM tiles and without the computing power. The other is slightly more fundamental - I agree that you can fairly easily calculate total climb over a way, but my question is whether that is really what you want? I think that most ways do not end at road junctions (certainly not the ones I have mapped anyway), but the routing programme will be interested in the total climb between road junctions, rather than for the total way? You are obviously right. What do you think about this idea: Instead of attaching tags to a way I could find all nodes which are shared by more than one way and then for each part of a way between two intersection create a relation like this: type=metainfo alt_difference=10.6 alt_total=27.3 distance=1032 with the way and the two intersection nodes as members. The distance is easily computed while doing the other processing. So the routing application could skip this step. If I am right then rather then your proposal will need closer integration with the routing programme - I think the router will pre-process the OSM file to calculate distance between junctions - we will want it to include altitudes at the same time. I explicitly tried to avoid a strong integration with the routers, because we have more than one and I expect even more in the future. Creating a library might be an option, if you think my idea is not really good the other way. However I still would prefer the preprocessor option since this allows to publish planet dumps which have the altitude information. When the processing is done by the router one has to publish data for each router seperatly or the user has to download the SRTM data. I want to make this application as userfriendly as possible and what could be more userfriendly than having a world file (or excerpt of it) which already includes the altitude data. I think your proposal to add a relation is an interesting one - it is essentially doing the same thing as the routing program will need to do for distances - you can imagine doing the same to calculate distance between junctions too, which is what the routing programs will do. I am still a little unsure how much benefit is gained by your proposed way of doing it, rather than directly incorporating it into a routing program - the routing program(s) will need to be modified to read your new relations anyway won't they? I would be inclined to suggest the two possibilities on your mailing list post to see what the others think. Of course the router will need some support for reading the data. But it won't need support for processing SRTM data directly. I'm also interested in some feedback on this point. What do the people on this mailinglist think about it? Regards, Hermann [1] http://r2d2.stefanm.com/soc2009/application2009_altitude.txt [2] http://www.uni-r.de/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Tiles not refreshing
Hi, I hope I've choosen right mailing list. Yesterday I edited part of map, mostly adding new roads. After over 24h slippy map doesn't contain my changes. Moreover I noticed that matching tile as dirty doesn't work. For example let's use: http://b.tile.openstreetmap.org/18/144049/86294.png http://openstreetmap.org/?lat=52.249444lon=17.821758zoom=18 Please see attached PNG to compare JOSM to slippy map. The road on slippy map is not updated and I can't match tile as dirty: wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/dirty 2/dev/null Tile submitted for rendering wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 -- Rafał Miłecki attachment: 8.png___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tiles not refreshing
On Wed, 2009-04-08 at 22:54 +0200, Rafał Miłecki wrote: Hi, I hope I've choosen right mailing list. Yesterday I edited part of map, mostly adding new roads. After over 24h slippy map doesn't contain my changes. Moreover I noticed that matching tile as dirty doesn't work. For example let's use: http://b.tile.openstreetmap.org/18/144049/86294.png http://openstreetmap.org/?lat=52.249444lon=17.821758zoom=18 Please see attached PNG to compare JOSM to slippy map. The road on slippy map is not updated and I can't match tile as dirty: wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/dirty 2/dev/null Tile submitted for rendering wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 The weekly reload of the Mapnik postgresql database is happening at the moment. This prevents any rendering or updates to the tile clean/dirty state until it is complete. The reload should be complete in a couple of hours but it may still take a day or two to render your changes since lots of tile will be queued for rendering. Jon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tiles not refreshing
W dniu 8 kwietnia 2009 23:10 użytkownik Jon Burgess jburgess...@googlemail.com napisał: On Wed, 2009-04-08 at 22:54 +0200, Rafał Miłecki wrote: I hope I've choosen right mailing list. Yesterday I edited part of map, mostly adding new roads. After over 24h slippy map doesn't contain my changes. Moreover I noticed that matching tile as dirty doesn't work. For example let's use: http://b.tile.openstreetmap.org/18/144049/86294.png http://openstreetmap.org/?lat=52.249444lon=17.821758zoom=18 Please see attached PNG to compare JOSM to slippy map. The road on slippy map is not updated and I can't match tile as dirty: wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/dirty 2/dev/null Tile submitted for rendering wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 The weekly reload of the Mapnik postgresql database is happening at the moment. This prevents any rendering or updates to the tile clean/dirty state until it is complete. The reload should be complete in a couple of hours but it may still take a day or two to render your changes since lots of tile will be queued for rendering. Thanks for explaining, I didn't know dirty-requesting is being ignored while global reloading. Sorry for noise :) -- Rafał Miłecki ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tiles not refreshing
On 8 Apr 2009, at 22:10, Jon Burgess wrote: On Wed, 2009-04-08 at 22:54 +0200, Rafał Miłecki wrote: Hi, I hope I've choosen right mailing list. Yesterday I edited part of map, mostly adding new roads. After over 24h slippy map doesn't contain my changes. Moreover I noticed that matching tile as dirty doesn't work. For example let's use: http://b.tile.openstreetmap.org/18/144049/86294.png http://openstreetmap.org/?lat=52.249444lon=17.821758zoom=18 Please see attached PNG to compare JOSM to slippy map. The road on slippy map is not updated and I can't match tile as dirty: wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/ status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/ dirty 2/dev/null Tile submitted for rendering wget -O - http://b.tile.openstreetmap.org/18/144049/86294.png/ status 2/dev/null Tile is clean. Last rendered at Wed Apr 08 10:25:30 2009 The weekly reload of the Mapnik postgresql database is happening at the moment. This prevents any rendering or updates to the tile clean/dirty state until it is complete. The reload should be complete in a couple of hours but it may still take a day or two to render your changes since lots of tile will be queued for rendering. As an interim solution you can use the minutely tile server, which has about a 10 minute delay on the osm data, and highlights the unnamed roads: http://matt.sandbox.cloudmade.com/?lat=52.249444lon=17.821758zoom=18 Shaun ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Server 0.6 doesn't answer ...
I've done a quick search through the server's log, and can find 401 and 500 responses both logged in the apache log. The rails log only shows 401 responses and not the 500s, so there's something odd happening between apache and rails. However, other requests to this method this evening have been functioning correctly according to the logs. 2009/4/8 Till Harbaum / Lists li...@harbaum.org: Hi, sending this request: PUT /api/0.6/changeset/create HTTP/1.1 Authorization: Basic x User-Agent: osm2go-libcurl/0.6.14 Host: api06.dev.openstreetmap.org Accept: */* Content-Length: 163 ?xml version=1.0 encoding=UTF-8? osm changeset tag k=created_by v=osm2go v0.6.14/ tag k=comment v=Kommentar/ /changeset /osm i either get an internal server error 500 or no reply at all. What is wrong with this request (despite the Auth i xxx'ed out)? Till ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- Regards, Thomas Wood (Edgemaster) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] spatial index - B-Tree over z-order curves vs R-Tree over GIST
Hi, I posted this question in the forum but I think that the list is more active. PostGIS uses R-Tree index over GIST. I'm trying to understand if it is possible to use couchDB for storing and indexing the osm data. couchdb is a schema less db for storing documents. Each document store data encoded as JSON. It uses B-Tree index so the only way I know to enable spatial index is to use space-filling-curves (z-order, morton codes) to translate a lat,lng to a number and then index all the numbers using a B-Tree. My question is why PostGIS choose to use R-Tree over GIST. Using z-order with B-Tree seem simpler and supported out of the box by most databases. Is there a significant performance difference or other issues against using z-order? http://couchdb.apache.org/ http://en.wikipedia.org/wiki/Z-order_(curve) http://postgis.refractions.net/documentation/manual-1.3/ch03.html#id2741805 Thanks ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] spatial index - B-Tree over z-order curves vs R-Tree over GIST
Pablo, pablo platt wrote: It uses B-Tree index so the only way I know to enable spatial index is to use space-filling-curves (z-order, morton codes) I've not come across these terms but from your references it seems that this is very much the same as our Quadtiles approach. The OSM db is still based on MySQL which has no multi-dimensional indexes either so we added a quadtree tile to each node which allows indexing. Since this happens on the application layer of course it makes our select queries quite ugly (where (tile x0 and tile x1) or (tile x2 and tile x3) or (...) ...) but they're not there to win a beauty contest and the parser hasn't complained. Yet. My question is why PostGIS choose to use R-Tree over GIST. I guess they had some reasons for it but I have no clue. One tends to assume that those who develop these things know what they do... until one day one looks at their source and finds comments that betray the fact that they're not any wiser than oneself ;) Using z-order with B-Tree seem simpler and supported out of the box by most databases. I don't think that the PostGIS developers cared about what was supported by other databases. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [GSoC] Automatic Street-Sign Detection and Reading - was: [GSoC] A GPX photo stamper.
Just a quick note to tell that I've updated my project proposal on the wiki: http://wiki.openstreetmap.org/wiki/GSoC_Student_Applications_2009#Draft_application:_Automatic_Street-Sign_Detection_and_Reading Since SIFT probably isn't going to work, we need something else. The color-histogram-matching might work. While going through my computer-vision book searching for ideas I found a number of other ideas (structural pattern recognition, texture-based features, more segmentation-oriented methods). More details and references in the proposal. I mean, come on, one of these methods must be able to detect street signs. :) - Tijs On Sat, Apr 4, 2009 at 3:38 PM, Stefan de Konink ste...@konink.de wrote: Matt Amos wrote: But please keep in mind that SIFT will *not* work for the typical streetsign because SIFT looks for features, which are typically not present in the 'easy to understand' signs (because they use colours). wouldn't SIFT (or variants thereof) work really well against a database of exemplars, i.e: pictures of existing street signs, isolated letters from the font used for street signage? http://sift.openstreetphoto.org/templates/C1.png This thing is totally 'unsiftable'. i'm happy to advise on this project, but i have to state up-front that i don't have the experience that stefan has with SIFT, etc... Just add your name to the list :) Stefan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] openstreetmap.org as target for spamming?
Dear maintainers of OSM, Usually we try to ignore spam, but now it seems, that openstreetmap.org has become some spam target (see below). Yours, Stefan - From: steeptso...@ua-news.net steeptso...@ua-news.net Date: 9. April 2009 01:58:10 GMT+02:00 To: AAA (a...@hsr.ch) a...@hsr.ch Subject: Kontaktformular ifs.hsr.ch Below is the result of your feedback form. It was submitted by (steeptso...@ua-news.net) on Thursday, April 9, 2009 at 01:58:10 --- Name: jeniorseplelo Vorname: Unsartticiasp Ort: jeniorseplelo Telefon: 123456 Nachricht: oioh , a href= http://www.openstreetmap.org/user/ZirabwynPoredric#1 hydrocodone order/a jhg$# , a href= http://www.openstreetmap.org/user/UlaynnorEthiredric#1 adipex/a mk8vgvg , a href= http://www.openstreetmap.org/user/LotherisOcili#1 ambien cr/a tgf$$ , a href= http://www.openstreetmap.org/user/LaeganYbeabard#1 buy ativan/a ddvyb efrgo , a href= http://www.openstreetmap.org/user/LothaokithFralewan#1 ephedra diet pill/a wsxsw , a href= http://www.openstreetmap.org/user/EreandAstoinad#1 generic xanax/a @@evdv , a href= http://www.openstreetmap.org/user/CirevJaewyth#1 alprazolam online/a iygy , a href= http://www.openstreetmap.org/user/AdwiratWaenwan#1 diazepam/a Absenden: Send --- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] JOSM version detect
Dirk Stöcker wrote: You should update. What do you think how I did compile the server stats? I generally do keep up with SVN, and that wasn't the problem. The problem was that I wasn't expecting to find the code to set the header in the file for the About box. I can see why it's been done that way, but it's still a bit ugly. It looks possible to move the generation of the version string to the build script and have the string attached to the JAR. I'll look into this, unless anyone has a violent objection? -- Jonathan (Jonobennett) ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
