Re: [OSM-talk] Osmxapi returns old data
Lars I think this is all sorted out now. Let me know if you suspect any other discrepancies. 80n On Fri, Mar 14, 2008 at 8:39 PM, Lars Aronsson [EMAIL PROTECTED] wrote: 80n wrote: Lars Can you point to a couple of specific instances please so that I can investigate further? If you try this very narrow region, wget ' http://www.informationfreeway.org/api/0.5/way[bbox=14.4,58.05,14.6,58.1][note=FIXMEhttp://www.informationfreeway.org/api/0.5/way%5Bbbox=14.4,58.05,14.6,58.1%5D%5Bnote=FIXMEpreviously unwayed segment]' -O mymap.osm You will get a 3280 bytes in mymap.osm, containing a few small untagged ways: way id='8173415' timestamp='2005-06-23T02:14:27Z' way id='8173417' timestamp='2005-06-23T02:14:30Z' way id='8173419' timestamp='2005-06-23T02:15:30Z' way id='8212509' timestamp='2006-02-24T18:11:40Z' way id='8212516' timestamp='2006-02-24T18:12:58Z' way id='8212518' timestamp='2006-02-24T18:14:36Z' way id='8212522' timestamp='2006-02-24T18:14:41Z' way id='8212526' timestamp='2006-02-24T18:15:07Z' Swedes will know this area as the ruins of Brahehus, near lake Vättern and motorway E4. I mapped this place with line segments in my first year of OSM, as shown by the timestamps above. Then Osmarender was introduced and the need to tag ways, and then someone added FIXME tags to old line segments. So just two weeks ago I edited these ways, combining them into larger units, adding the appropriate tags and removing the note=FIXME tags. When you open mymap.osm with JOSM, you will notice that all ways are 2 nodes (a single line segment) and they are not connected, but sprinkled out. If you download fresh data from OSM (without changing the zoom), you will notice that all FIXME tags are gone, and the ways are larger connected units. The last way in the list above, having id=8212526, still exists, but now is now a 4 node way (highway=motorway_link). It was tagged fixme in the planet.osm dump of 080227, but not in 080305 or 080312. At first I thought that 2-node ways that I had removed were lingering in osmxapi, but that doesn't seem to be the case. I guess the neighboring ways 8212523-8212525 were correctly removed. But the way that remained after my combination and tagging, 8212526, apparently wasn't modified in osmxapi. -- Lars Aronsson ([EMAIL PROTECTED]) Aronsson Datateknik - http://aronsson.se ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
On Sat, Mar 15, 2008 at 1:43 AM, 80n [EMAIL PROTECTED] wrote: On Sat, Mar 15, 2008 at 12:46 AM, Jon Burgess [EMAIL PROTECTED] wrote: On Sat, 2008-03-15 at 00:30 +, 80n wrote: Lars Thanks for this info, its been very useful. Some edits, that were made during the period when the incremental feeds were down, have not been resynchronized correctly. The problem is due to a difference in the timestamp format between the planet dump and the incremental feeds. I'll fix the resync script and re-run it. 80n I thought they both used the same format:- $ gzip -dc /home/www/tile/direct/planet/planet-080312.osm.gz | grep timestamp | head -n 3 node id=2 lat=50.1360074 lon=8.3023717 timestamp=2007-10-04T18:28:37Z node id=4 lat=51.5220733 lon=-0.1458135 timestamp=2007-01-29T08:48:14Z/ node id=11 lat=4.0840331 lon=73.5129514 timestamp=2006-11-04T18:15:03Z $ bzip2 -dc daily-20080307-20080308.osc.bz2 | grep timestamp | head -n 3 node id=122329 timestamp=2008-03-07T07:33:53Z user=mbuege lat=53.5374636 lon=10.0309499/ node id=122330 timestamp=2008-03-07T07:43:48Z user=mbuege lat=53.5369426 lon=10.0339214/ node id=122331 timestamp=2008-03-07T07:27:15Z user=mbuege lat=53.5389492 lon=10.0325114/ I can see the order of the attributes is different and the diffs contain the user, but the timestamp looks the same format to me. Or do you mean the timestamp in the filename? Yeah, the problem is that there _used_ to be a difference, but now there isn't. My script was expecting them to be different. On further inspection, what I said above is complete rubbish. The timestamp format difference is between the the planet dump and the API not Osmosis: Planet dump: node id=2 lat=50.1360074 lon=8.3023717 timestamp=2007-10-04T18:28:37Z API: node id=2 lat=50.1360074 lon=8.3023717 timestamp=2007-10-04T19:28:37+01:00 My code, which is pants, wrongly assumes that Osmosis was using a timestamp like the API produces, when in fact its the same as the planet dump. Maybe I'll be able to fix it properly now I know what I'm talking about. 80n Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
On Sat, 2008-03-15 at 07:31 +, 80n wrote: On further inspection, what I said above is complete rubbish. The timestamp format difference is between the the planet dump and the API not Osmosis: Planet dump: node id=2 lat=50.1360074 lon=8.3023717 timestamp=2007-10-04T18:28:37Z API: node id=2 lat=50.1360074 lon=8.3023717 timestamp=2007-10-04T19:28:37+01:00 My code, which is pants, wrongly assumes that Osmosis was using a timestamp like the API produces, when in fact its the same as the planet dump. Maybe I'll be able to fix it properly now I know what I'm talking about. The timestamps in the planet dumps did match the API until a few months ago. Brett and I agreed to converge on a common timestamp format since the generic Java date parsing was horribly slow. I suppose we could change the API to use this same format if it would help other tools. Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
Karl Newman wrote: This is probably due to the recently broken (and now fixed) daily/hourly diffs. After this week's planet dump is imported (probably today), the hourly diffs should be able to properly keep it up to date as the wiki page states. Yes, it's much better now. But it's not perfect. I still get many (albeit not *so* many) small (two node) ways from Osmxapi that I know have been removed (long ago) in the OSM database. When I grab an area of FIXME with Osmxapi, open this .osm file in JOSM, and download fresh data from OSM, the FIXME disappear. -- Lars Aronsson ([EMAIL PROTECTED]) Aronsson Datateknik - http://aronsson.se ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
Lars Can you point to a couple of specific instances please so that I can investigate further? Osmxapi was resynchronized against this week's planet file ( planet-080312.osm) plus the diff files from then onwards. So if there are still discrepancies all the evidence is still be available at the moment to track down how this is happening. 80n On Fri, Mar 14, 2008 at 5:22 PM, Lars Aronsson [EMAIL PROTECTED] wrote: Karl Newman wrote: This is probably due to the recently broken (and now fixed) daily/hourly diffs. After this week's planet dump is imported (probably today), the hourly diffs should be able to properly keep it up to date as the wiki page states. Yes, it's much better now. But it's not perfect. I still get many (albeit not *so* many) small (two node) ways from Osmxapi that I know have been removed (long ago) in the OSM database. When I grab an area of FIXME with Osmxapi, open this .osm file in JOSM, and download fresh data from OSM, the FIXME disappear. -- Lars Aronsson ([EMAIL PROTECTED]) Aronsson Datateknik - http://aronsson.se ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
80n wrote: Lars Can you point to a couple of specific instances please so that I can investigate further? If you try this very narrow region, wget 'http://www.informationfreeway.org/api/0.5/way[bbox=14.4,58.05,14.6,58.1][note=FIXME previously unwayed segment]' -O mymap.osm You will get a 3280 bytes in mymap.osm, containing a few small untagged ways: way id='8173415' timestamp='2005-06-23T02:14:27Z' way id='8173417' timestamp='2005-06-23T02:14:30Z' way id='8173419' timestamp='2005-06-23T02:15:30Z' way id='8212509' timestamp='2006-02-24T18:11:40Z' way id='8212516' timestamp='2006-02-24T18:12:58Z' way id='8212518' timestamp='2006-02-24T18:14:36Z' way id='8212522' timestamp='2006-02-24T18:14:41Z' way id='8212526' timestamp='2006-02-24T18:15:07Z' Swedes will know this area as the ruins of Brahehus, near lake Vättern and motorway E4. I mapped this place with line segments in my first year of OSM, as shown by the timestamps above. Then Osmarender was introduced and the need to tag ways, and then someone added FIXME tags to old line segments. So just two weeks ago I edited these ways, combining them into larger units, adding the appropriate tags and removing the note=FIXME tags. When you open mymap.osm with JOSM, you will notice that all ways are 2 nodes (a single line segment) and they are not connected, but sprinkled out. If you download fresh data from OSM (without changing the zoom), you will notice that all FIXME tags are gone, and the ways are larger connected units. The last way in the list above, having id=8212526, still exists, but now is now a 4 node way (highway=motorway_link). It was tagged fixme in the planet.osm dump of 080227, but not in 080305 or 080312. At first I thought that 2-node ways that I had removed were lingering in osmxapi, but that doesn't seem to be the case. I guess the neighboring ways 8212523-8212525 were correctly removed. But the way that remained after my combination and tagging, 8212526, apparently wasn't modified in osmxapi. -- Lars Aronsson ([EMAIL PROTECTED]) Aronsson Datateknik - http://aronsson.se ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
Lars Thanks for this info, its been very useful. Some edits, that were made during the period when the incremental feeds were down, have not been resynchronized correctly. The problem is due to a difference in the timestamp format between the planet dump and the incremental feeds. I'll fix the resync script and re-run it. 80n On Fri, Mar 14, 2008 at 8:39 PM, Lars Aronsson [EMAIL PROTECTED] wrote: 80n wrote: Lars Can you point to a couple of specific instances please so that I can investigate further? If you try this very narrow region, wget ' http://www.informationfreeway.org/api/0.5/way[bbox=14.4,58.05,14.6,58.1][note=FIXMEhttp://www.informationfreeway.org/api/0.5/way%5Bbbox=14.4,58.05,14.6,58.1%5D%5Bnote=FIXMEpreviously unwayed segment]' -O mymap.osm You will get a 3280 bytes in mymap.osm, containing a few small untagged ways: way id='8173415' timestamp='2005-06-23T02:14:27Z' way id='8173417' timestamp='2005-06-23T02:14:30Z' way id='8173419' timestamp='2005-06-23T02:15:30Z' way id='8212509' timestamp='2006-02-24T18:11:40Z' way id='8212516' timestamp='2006-02-24T18:12:58Z' way id='8212518' timestamp='2006-02-24T18:14:36Z' way id='8212522' timestamp='2006-02-24T18:14:41Z' way id='8212526' timestamp='2006-02-24T18:15:07Z' Swedes will know this area as the ruins of Brahehus, near lake Vättern and motorway E4. I mapped this place with line segments in my first year of OSM, as shown by the timestamps above. Then Osmarender was introduced and the need to tag ways, and then someone added FIXME tags to old line segments. So just two weeks ago I edited these ways, combining them into larger units, adding the appropriate tags and removing the note=FIXME tags. When you open mymap.osm with JOSM, you will notice that all ways are 2 nodes (a single line segment) and they are not connected, but sprinkled out. If you download fresh data from OSM (without changing the zoom), you will notice that all FIXME tags are gone, and the ways are larger connected units. The last way in the list above, having id=8212526, still exists, but now is now a 4 node way (highway=motorway_link). It was tagged fixme in the planet.osm dump of 080227, but not in 080305 or 080312. At first I thought that 2-node ways that I had removed were lingering in osmxapi, but that doesn't seem to be the case. I guess the neighboring ways 8212523-8212525 were correctly removed. But the way that remained after my combination and tagging, 8212526, apparently wasn't modified in osmxapi. -- Lars Aronsson ([EMAIL PROTECTED]) Aronsson Datateknik - http://aronsson.se ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
On Sat, 2008-03-15 at 00:30 +, 80n wrote: Lars Thanks for this info, its been very useful. Some edits, that were made during the period when the incremental feeds were down, have not been resynchronized correctly. The problem is due to a difference in the timestamp format between the planet dump and the incremental feeds. I'll fix the resync script and re-run it. 80n I thought they both used the same format:- $ gzip -dc /home/www/tile/direct/planet/planet-080312.osm.gz | grep timestamp | head -n 3 node id=2 lat=50.1360074 lon=8.3023717 timestamp=2007-10-04T18:28:37Z node id=4 lat=51.5220733 lon=-0.1458135 timestamp=2007-01-29T08:48:14Z/ node id=11 lat=4.0840331 lon=73.5129514 timestamp=2006-11-04T18:15:03Z $ bzip2 -dc daily-20080307-20080308.osc.bz2 | grep timestamp | head -n 3 node id=122329 timestamp=2008-03-07T07:33:53Z user=mbuege lat=53.5374636 lon=10.0309499/ node id=122330 timestamp=2008-03-07T07:43:48Z user=mbuege lat=53.5369426 lon=10.0339214/ node id=122331 timestamp=2008-03-07T07:27:15Z user=mbuege lat=53.5389492 lon=10.0325114/ I can see the order of the attributes is different and the diffs contain the user, but the timestamp looks the same format to me. Or do you mean the timestamp in the filename? Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
On Thu, Mar 13, 2008 at 5:26 AM, Karl Newman [EMAIL PROTECTED] wrote: On Wed, Mar 12, 2008 at 9:30 PM, Lars Aronsson [EMAIL PROTECTED] wrote: Today I learned about [[Osmxapi]] and found it really useful for digging out ways tagged with FIXME. However, on the wiki page it says the data is never more than about 2 hours old and this clearly doesn't hold, as I got data from InformationFreeway that was changed in OSM many days ago. What's up? In particular, this command: wget ' http://www.informationfreeway.org/api/0.5/way[bbox=15,58,15.5,59][note=FIXMEhttp://www.informationfreeway.org/api/0.5/way%5Bbbox=15,58,15.5,59%5D%5Bnote=FIXMEpreviously unwayed segment]' -O mymap.osm returns 1.5 megabytes of data with a mesh of single line segment ways tagged with FIXME, that I last week have combined into longer ways tagged with highway=secondary. This is probably due to the recently broken (and now fixed) daily/hourly diffs. After this week's planet dump is imported (probably today), the hourly diffs should be able to properly keep it up to date as the wiki page states. This *is* due to the recently broken (and now fixed) daily/hourly diffs. There's a resync that was started about 9 hours ago. 80n Karl ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
[OSM-talk] Osmxapi returns old data
Today I learned about [[Osmxapi]] and found it really useful for digging out ways tagged with FIXME. However, on the wiki page it says the data is never more than about 2 hours old and this clearly doesn't hold, as I got data from InformationFreeway that was changed in OSM many days ago. What's up? In particular, this command: wget 'http://www.informationfreeway.org/api/0.5/way[bbox=15,58,15.5,59][note=FIXME previously unwayed segment]' -O mymap.osm returns 1.5 megabytes of data with a mesh of single line segment ways tagged with FIXME, that I last week have combined into longer ways tagged with highway=secondary. -- Lars Aronsson ([EMAIL PROTECTED]) Aronsson Datateknik - http://aronsson.se ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmxapi returns old data
On Wed, Mar 12, 2008 at 9:30 PM, Lars Aronsson [EMAIL PROTECTED] wrote: Today I learned about [[Osmxapi]] and found it really useful for digging out ways tagged with FIXME. However, on the wiki page it says the data is never more than about 2 hours old and this clearly doesn't hold, as I got data from InformationFreeway that was changed in OSM many days ago. What's up? In particular, this command: wget ' http://www.informationfreeway.org/api/0.5/way[bbox=15,58,15.5,59][note=FIXMEhttp://www.informationfreeway.org/api/0.5/way%5Bbbox=15,58,15.5,59%5D%5Bnote=FIXMEpreviously unwayed segment]' -O mymap.osm returns 1.5 megabytes of data with a mesh of single line segment ways tagged with FIXME, that I last week have combined into longer ways tagged with highway=secondary. This is probably due to the recently broken (and now fixed) daily/hourly diffs. After this week's planet dump is imported (probably today), the hourly diffs should be able to properly keep it up to date as the wiki page states. Karl ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
