Re: [OSM-talk] Islands that will vanish in the license change
- Original Message - From: Toby Murray toby.mur...@gmail.com To: OSM Talk talk@openstreetmap.org Sent: Friday, April 13, 2012 7:14 AM Subject: [OSM-talk] Islands that will vanish in the license change Thanks to Paul Norman's efforts and visualizations based on it[1], I've now added to the CT-only coastline error checker map [1] my estimation of island ways which might be at risk of deletion. I've tried to identify all ways taged with natural = coastline, created by someone who has not agreed to the CT's, and which are not marked odbl=clean . I'm afraid that dispalying this number of points has impacted performance when viewing the map in IE. As Toby reported, most of the problematic ways are around Australia. (Note that although the island points layer is based on this mornings' data, other error layers are based on data from 19 April) David there has been a lot of activity in remapping coastlines lately and a lot of improvement. However one loophole that Paul's method does not detect is islands that will have their coastlines vanish completely. I decided to take a look at this tonight. So far I have come up with a pretty hackish way of looking at things... but I think it might still be of use. I downloaded all natural=coastline ways from my jxapi. Then I split the world into 4 parts to make them small enough for me to open in JOSM. Then I selected all objects that were last touched by an accepting user and purged them from the data set. What is left is all ways that were last touched by a decliner. Some of these are actually OK from a license standpoint. Maybe the decliner just deleted a tag or added some nodes in the middle of a way which will obviously distort the geometry but the coastline topology will remain intact so they don't show up in Paul's files. Also, some things that I purged may still be heavily impacted or even completely removed by the license change if they were created by a decliner but last touched by someone else. So it isn't perfect. The south/west quadrant of the world is actually pretty much good to go. I already fixed a few islands. The north/west one is still a little large so I may have to do some more tinkering there. The one I have ready to go right now is south/east (Australia) and I saw this topic come up in the talk-au archives a few days ago so I thought I would go ahead and share. Perhaps someone who is subscribed to talk-au can forward this? What I have is a ~10MB .osm file containing 537 ways (plus some stray nodes that should just be ignored): http://ni.kwsn.net/~toby/OSM/coastline_SE_bad.osm.gz This file covers from the equator to the south pole and from 0 to 180 longitude so it is more than Just Australia although that is the most impacted area. The way to use this would be to download it and open it in JOSM. Then do a type:way search and run the license plugin on that. Then just look for the big red blobs. DO NOT use this layer to edit and upload or terrible things are likely to happen. Use it only as a guide to find trouble spots. I set the upload=false flag in the file so JOSM should be very clear about this if you try to upload from this layer. So download a problem area to a new layer and replace dirty coastlines to your heart's content. The biggest blob of red is on the northeast side of Australia. Some of them are random rocks along the coastline that cover a few square meters. Some are large islands. Unfortunately it looks like Bing isn't good enough to retrace some of these but I'm hoping the locals may have other sources. Enjoy, Toby [1] http://www.wightpaths.co.uk/coast/CT-only.php ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Islands that will vanish in the license change
Thanks to Paul Norman's efforts and visualizations based on it[1], there has been a lot of activity in remapping coastlines lately and a lot of improvement. However one loophole that Paul's method does not detect is islands that will have their coastlines vanish completely. I decided to take a look at this tonight. So far I have come up with a pretty hackish way of looking at things... but I think it might still be of use. I downloaded all natural=coastline ways from my jxapi. Then I split the world into 4 parts to make them small enough for me to open in JOSM. Then I selected all objects that were last touched by an accepting user and purged them from the data set. What is left is all ways that were last touched by a decliner. Some of these are actually OK from a license standpoint. Maybe the decliner just deleted a tag or added some nodes in the middle of a way which will obviously distort the geometry but the coastline topology will remain intact so they don't show up in Paul's files. Also, some things that I purged may still be heavily impacted or even completely removed by the license change if they were created by a decliner but last touched by someone else. So it isn't perfect. The south/west quadrant of the world is actually pretty much good to go. I already fixed a few islands. The north/west one is still a little large so I may have to do some more tinkering there. The one I have ready to go right now is south/east (Australia) and I saw this topic come up in the talk-au archives a few days ago so I thought I would go ahead and share. Perhaps someone who is subscribed to talk-au can forward this? What I have is a ~10MB .osm file containing 537 ways (plus some stray nodes that should just be ignored): http://ni.kwsn.net/~toby/OSM/coastline_SE_bad.osm.gz This file covers from the equator to the south pole and from 0 to 180 longitude so it is more than Just Australia although that is the most impacted area. The way to use this would be to download it and open it in JOSM. Then do a type:way search and run the license plugin on that. Then just look for the big red blobs. DO NOT use this layer to edit and upload or terrible things are likely to happen. Use it only as a guide to find trouble spots. I set the upload=false flag in the file so JOSM should be very clear about this if you try to upload from this layer. So download a problem area to a new layer and replace dirty coastlines to your heart's content. The biggest blob of red is on the northeast side of Australia. Some of them are random rocks along the coastline that cover a few square meters. Some are large islands. Unfortunately it looks like Bing isn't good enough to retrace some of these but I'm hoping the locals may have other sources. Enjoy, Toby [1] http://www.wightpaths.co.uk/coast/CT-only.php ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Islands that will vanish in the license change
Hi, On 04/13/2012 08:14 AM, Toby Murray wrote: I downloaded all natural=coastline ways from my jxapi. Then I split the world into 4 parts to make them small enough for me to open in JOSM. Then I selected all objects that were last touched by an accepting user and purged them from the data set. What is left is all ways that were last touched by a decliner. [...] It is possible that the procedure outlined in this posting might also be of use: http://lists.openstreetmap.org/pipermail/talk-gb/2012-April/013059.html Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Islands that will vanish in the license change
From: Frederik Ramm [mailto:frede...@remote.org] Subject: Re: [OSM-talk] Islands that will vanish in the license change Hi, On 04/13/2012 08:14 AM, Toby Murray wrote: I downloaded all natural=coastline ways from my jxapi. Then I split the world into 4 parts to make them small enough for me to open in JOSM. Then I selected all objects that were last touched by an accepting user and purged them from the data set. What is left is all ways that were last touched by a decliner. [...] It is possible that the procedure outlined in this posting might also be of use: http://lists.openstreetmap.org/pipermail/talk-gb/2012-April/013059.html This technique works well for smaller areas, but when I tried running the license check plugin on a 5 GB .osm file it was unhappy. I have all the code written to do this automatically except the logic, which I'm finding oddly difficult. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Islands that will vanish in the license change
Hi, On 04/13/12 11:00, Paul Norman wrote: This technique works well for smaller areas, but when I tried running the license check plugin on a 5 GB .osm file it was unhappy. With very large files, I do manage to get a result from the plugin but JOSM's performance rapidly deteriorates (no doubt due to my hapharzardly implementing the plugin because as soon as I switch off the license check layer things work smoothly again). In these cases I tend to: 1. load very large file 2. run license check 3. visually identify problem area 4. drop license check result layer 5. select (one of a number of) problem area(s) 6. invert selection 7. purge data 8. re-run license check That way, I can zoom in on problem areas without always having everything loaded. I have all the code written to do this automatically except the logic, which I'm finding oddly difficult. In addition to the logic in *my* code being difficult, it is also not the ultimate algorithm; while there will be little difference between the WTFE algorithm/license change plugin and the actual license change bot, there may be situations where they make different decisions. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Islands that will vanish in the license change
From: Frederik Ramm [mailto:frede...@remote.org] Subject: Re: [OSM-talk] Islands that will vanish in the license change Hi, On 04/13/12 11:00, Paul Norman wrote: This technique works well for smaller areas, but when I tried running the license check plugin on a 5 GB .osm file it was unhappy. With very large files, I do manage to get a result from the plugin but JOSM's performance rapidly deteriorates (no doubt due to my hapharzardly implementing the plugin because as soon as I switch off the license check layer things work smoothly again). In these cases I tend to: 1. load very large file 2. run license check 3. visually identify problem area 4. drop license check result layer 5. select (one of a number of) problem area(s) 6. invert selection 7. purge data 8. re-run license check That way, I can zoom in on problem areas without always having everything loaded. I have all the code written to do this automatically except the logic, which I'm finding oddly difficult. In addition to the logic in *my* code being difficult, it is also not the ultimate algorithm; while there will be little difference between the WTFE algorithm/license change plugin and the actual license change bot, there may be situations where they make different decisions. So I'm running difficult logic on a service that has difficult logic itself? No wonder I can't always predict the results :) The biggest problem I ran into when implementing cleanway was that I didn't have a full history of the objects. If an acceptor creates a way and a decliner adds natural=coastline to it WTFE reports the same as if an acceptor creates a natural=coastline way and a decliner adds some other tag, but the two scenarios are very different to the coastline process. My main goal was to identify the big problem areas (e.g great lakes, west coast of the US) and it has worked for that. Post-rebuild it looks like any coastline breakages will be fairly isolated and not flood large areas. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
