[OSM-dev] GSoC Proposal: OSM Dynamic Attributes Linter
Hello! I'm Wisnu Adi Nurcahyo, an Informatics Engineering student from Telkom University at Bandung, Indonesia. I'm interested to join GSoC in OpenStreetMap organization. I've submitted a proposal draft titled "OSM Dynamic Attributes Linter" which didn't have a mentor for me. Please review my proposal. I'm really glad if someone would be my mentor. In case you can't see the proposal, I will writes the abstract here. Abstract - Sometimes contributor didn't follows the naming convention in their country and ignore it. Or, some contributors didn't know the naming convention in it's country. We need dynamic linter with custom presets to solves this problem. With this linter, everyone may writes their own rules in the presets. This mean every rules maybe created and standardized. Then, OSM attributes will be clean. Motivation --- In Indonesia, especially Kalimantan. There are many "bad" data. An example is an attribute of SMAN 1 Bintang Ara ( http://www.openstreetmap.org/way/400611340). It's a Senior High School. But, the "school:type_idn" had an attribute is Elementary School and didn’t have an address attribute. Not only in Kalimantan, Papua also had a same problem. An example is an attribute of this data (http://www.openstreetmap.org/way/183960297) in Papua. The “admin_level” attribute must be a number. But, it’s not a number. The “addr:full” attribute also have a bad data. So, depend on this problem, I can build rules and writes a presets which will improve data quality for every places in Indonesia, and the world. Thanks! :D ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Natural Earth
On Wednesday 29 March 2017, Sebastian Kürten wrote: > Can somebody tell me whether the map on openstreetmap.org still > relies on Natural Earth data for rendering of lower zoom levels? Natural Earth data at the moment is used for the lowest zoom level boundaries (z1/2/3) and for the builtup areas at z8/9. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
No need to pick nits Martin, you know what I mean. Any lat/lon associated with a non-geographic postcode is arbitrary and volatile. It is not needed for delivery purposes - only the code itself is used. It can be "moved" to some other location, possibly a long distance away, in a way that is not possible with a normal postcode. In the same way as a mobile or service (0800 etc) phone number might have a "lat/lon" representing the current home address of its "user", but that is not the same as the lat/lon of a fixed line. On 2017-03-29 13:32, Martin Koppenhoefer wrote: > 2017-03-29 12:14 GMT+02:00 Colin Smale : > >> There are also in some countries "non-geographic postal codes" - things like >> reply numbers and PO Boxes. > > they are geographic as well, just that their place is at the post box and not > at the owner of that box... ;-) > > Cheers, > Martin___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
2017-03-29 12:14 GMT+02:00 Colin Smale : > There are also in some countries "non-geographic postal codes" - things > like reply numbers and PO Boxes. they are geographic as well, just that their place is at the post box and not at the owner of that box... ;-) Cheers, Martin ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
There are also in some countries "non-geographic postal codes" - things like reply numbers and PO Boxes. If we are able to filter these out of the data in some way it may make the job a little easier. Although the UK postcodes are not defined as a boundary but as a list of points, there are several "reverse geocoding" services in the UK which implicitly will allow you to find the boundary that their algorithm+data leads to. I have no idea how "accurate" their results are for a random point, or indeed, how you would measure that accuracy. //colin On 2017-03-29 10:51, Jo wrote: > I'm in Belgium, so I'm mostly familiar with postal codes here. There are some > oddities, but mostly they are not too illogical and it is possible to draw > polygons around/in between them. > It seems Alex already did this exercise for Austria and Switzerland, so I > think it's possible there as well. He'll probably needs to talk to the > mailing lists for each country separately to figure out whether there is > willingness to define (initial versions of) these boundaries. > > Polyglot > > 2017-03-29 10:46 GMT+02:00 Tom Hughes : > On 29/03/17 09:41, Jo wrote: > > For postal_code boundaries, they will very often follow existing > boundaries, except where they don't... so I would say it is possible to > draw them by mostly following the existing admin_boundaries. So now you > appear to be talking about the UK which I do know about and which definitely > doesn't have boundaries as such. > > Royal Mail as I understand it defines each post code by a list of addresses. > They do also provide a centroid point derived from that list but I don't > believe they provide any sort of boundary. > > Tom > > -- > Tom Hughes (t...@compton.nu) > http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Natural Earth
Can somebody tell me whether the map on openstreetmap.org still relies on Natural Earth data for rendering of lower zoom levels? Thanks, Sebastian ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
The Netherlands are a very strange beast concerning postal codes... :-) Definitely an example where such an approach wouldn't make sense. OTOH, you have complete address data for every single building in The Netherlands, so there is also no real need for it (anymore). Polyglot 2017-03-29 11:07 GMT+02:00 Maarten Deen : > On 2017-03-29 10:20, Tom Hughes wrote: > > Rather they are just defined by a list of addresses, being a set of >> addresses that are a convenient group to deliver to. >> >> Now obviously you can draw any number of shapes around those addresses >> but none of those shapes is in any way an official or definitive >> boundary for the postal code. >> > > True, and depending on the level of detail of the postal code this will be > very artificial boundary in the Netherlands. > The number part of a dutch postal code would in most cases result into a > proper boundary, but if you go to the number+letter part, wou will include > streets in the area that do not belong to that postal code, but will also > not be in the shape of the proper postal code because there are no houses > there. The street behind my house will be one such case. > > So, you will be able to find the proper postal code beloning to a house, > but given a certain postal code, you could get directions to the wrong > location. > > Regards, > Maarten > > > > > ___ > dev mailing list > dev@openstreetmap.org > https://lists.openstreetmap.org/listinfo/dev > ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
On 2017-03-29 10:20, Tom Hughes wrote: Rather they are just defined by a list of addresses, being a set of addresses that are a convenient group to deliver to. Now obviously you can draw any number of shapes around those addresses but none of those shapes is in any way an official or definitive boundary for the postal code. True, and depending on the level of detail of the postal code this will be very artificial boundary in the Netherlands. The number part of a dutch postal code would in most cases result into a proper boundary, but if you go to the number+letter part, wou will include streets in the area that do not belong to that postal code, but will also not be in the shape of the proper postal code because there are no houses there. The street behind my house will be one such case. So, you will be able to find the proper postal code beloning to a house, but given a certain postal code, you could get directions to the wrong location. Regards, Maarten ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
I'm in Belgium, so I'm mostly familiar with postal codes here. There are some oddities, but mostly they are not too illogical and it is possible to draw polygons around/in between them. It seems Alex already did this exercise for Austria and Switzerland, so I think it's possible there as well. He'll probably needs to talk to the mailing lists for each country separately to figure out whether there is willingness to define (initial versions of) these boundaries. Polyglot 2017-03-29 10:46 GMT+02:00 Tom Hughes : > On 29/03/17 09:41, Jo wrote: > > For postal_code boundaries, they will very often follow existing >> boundaries, except where they don't... so I would say it is possible to >> draw them by mostly following the existing admin_boundaries. >> > > So now you appear to be talking about the UK which I do know about and > which definitely doesn't have boundaries as such. > > Royal Mail as I understand it defines each post code by a list of > addresses. They do also provide a centroid point derived from that list but > I don't believe they provide any sort of boundary. > > > Tom > > -- > Tom Hughes (t...@compton.nu) > http://compton.nu/ > ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
On 29/03/17 09:41, Jo wrote: For postal_code boundaries, they will very often follow existing boundaries, except where they don't... so I would say it is possible to draw them by mostly following the existing admin_boundaries. So now you appear to be talking about the UK which I do know about and which definitely doesn't have boundaries as such. Royal Mail as I understand it defines each post code by a list of addresses. They do also provide a centroid point derived from that list but I don't believe they provide any sort of boundary. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
What I did, at some point when I was trying to figure out zones of bus stops, was to create a MapCSS style which gave me different background colours for each numbered zone. This helped me to visually find the 'outliers', based on the ones that I did already figured out the zone for. For postal_code boundaries, they will very often follow existing boundaries, except where they don't... so I would say it is possible to draw them by mostly following the existing admin_boundaries. If we ever want to draw parishes, we'll probably have to 'wing it' in a comparable way. To me this feels like how we started drawing everything all the way back when OpenStreetMap was a blank canvas. We start with something, if it's wrong we correct it and slowly but surely we get to a point where we have the best data. And possibly OSM becomes the only place where those shapes can be found. It's unlikely they are exactly what the post offices use, but it's even more unlikely that the post offices will ever share them in a way we can use them. And if all the houses with that post code are within them, they are 'good enough'. Saying that you should contribute them to Nominatim might be 'the right thing to do', but then we lose the possibility to improve them incrementally, which is exactly what OpenStreetMap excels in. Polyglot 2017-03-29 10:20 GMT+02:00 Tom Hughes : > On 29/03/17 08:58, Frederik Ramm wrote: > > On 29.03.2017 09:10, Alex K wrote: >> >>> * For one, this type of information is already part of >>> OSM: http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dpostal_code >>> >> >> Generally we don't like data that is impossible (or difficult e.g. >> "knocking on doors") to verify on the ground, but we do make exceptions >> for admin boundaries and, usually, postal code boundaries. >> > > If the postal code boundary is a genuine thing that exists then sure. > > I don't know about the countries in question, but in the countries that I > do know about there is no such thing as a postal code boundary because the > authority that assigns postal codes doesn't do so using geographic areas > like that, and doesn't special any specific boundary. > > Rather they are just defined by a list of addresses, being a set of > addresses that are a convenient group to deliver to. > > Now obviously you can draw any number of shapes around those addresses but > none of those shapes is in any way an official or definitive boundary for > the postal code. > > Tom > > -- > Tom Hughes (t...@compton.nu) > http://compton.nu/ > > ___ > dev mailing list > dev@openstreetmap.org > https://lists.openstreetmap.org/listinfo/dev > ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
On 29/03/17 08:58, Frederik Ramm wrote: On 29.03.2017 09:10, Alex K wrote: * For one, this type of information is already part of OSM: http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dpostal_code Generally we don't like data that is impossible (or difficult e.g. "knocking on doors") to verify on the ground, but we do make exceptions for admin boundaries and, usually, postal code boundaries. If the postal code boundary is a genuine thing that exists then sure. I don't know about the countries in question, but in the countries that I do know about there is no such thing as a postal code boundary because the authority that assigns postal codes doesn't do so using geographic areas like that, and doesn't special any specific boundary. Rather they are just defined by a list of addresses, being a set of addresses that are a convenient group to deliver to. Now obviously you can draw any number of shapes around those addresses but none of those shapes is in any way an official or definitive boundary for the postal code. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
Hi, On 29.03.2017 09:10, Alex K wrote: > * For one, this type of information is already part of > OSM: http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dpostal_code Generally we don't like data that is impossible (or difficult e.g. "knocking on doors") to verify on the ground, but we do make exceptions for admin boundaries and, usually, postal code boundaries. But that exception would certainly not apply to derived data; if it is desired to use derived data in geocoding, then the code to run the derivation must be in Nominatim (so that any derived geometries automatically update when base data is modified). > * The current postal code tagging of admin levels in > Austria/Switzerland is not only of bad quality but also wrong from a > logical aspect. The boundaries of the two have no connection in real > life. We should get rid of THAT information because it produces > false results in Nominatim. You are welcome to suggest the deletion of this information on the mailing lists/forums in Austria and Switzerland, and remove them if the community agrees. > So the information has high practicle value and can help push OSM into > new areas of usage. That's why I believe it is very important to add it > for more countries... You can add code that does sophisticated post code guessing to Nominatim but you cannot add the result of a sophisticated guessing algorithm as a base geometry in OSM - even less so if the algorithm you used for guessing isn't available for inspection. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
Hi Tom, I see your point. Hope you don't mind if I make a quick counter argument and try to convice you: For one, this type of information is already part of OSM: http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dpostal_code According to my evaluations, the postal code relation coverage in Germany is near perfect (less than 1% error according to my evaluations, depending on what you measure) without (as far as I know) there being publically available information. The current postal code tagging of admin levels in Austria/Switzerland is not only of bad quality but also wrong from a logical aspect. The boundaries of the two have no connection in real life. We should get rid of THAT information because it produces false results in Nominatim. For most countries, there seem to be no official, publically available data You can verify it if you a) have sets of addresses (e.g. customer databases) that are so large that incorrect entries can be filtered out as noise or b) drive along the boundary, knock on doors and ask people what their postal address is (no serious suggestion of course, but technically not much worse then when people walk to each individual house to map the house number) It's actually not "invented" but rather (imperfect) "derived" data, I think that's an important distinction. By having a high quality initial reconstruction, it makes it easy for others in the community to find errors in individual boundary lines and produce even better postal code areas. This information helped me detect/correct incorrect tagging on individual houses because of the inconsistency between postal code areas and the invidiual node/way. Vice versa, each new house that is tagged with a postal code will make it possible to detect inconsistencies and improve the postal code relation. Add to that the improvement to Novinatim if we would have that information in OSM. So the information has high practicle value and can help push OSM into new areas of usage. That's why I believe it is very important to add it for more countries... Alex Tom Hughes wrote on 29.03.2017 00:00: On 28/03/17 22:24, Alex K wrote: > Basically I'm using a semi-automatic process which takes all the know > data points (e.g. buildings/nodes with an explicit postal code tagged to > them) in OSM, generate voronoi cells and then merge them to larger > regions. Then I do manually editing and clean up in my tool (aligning > boundaries more nicely, reducing number of nodes, etc) and finally want > to import the results as new relations into OSM. I did some evaluations > on Austria against a real life data set and although the computed > regions can only be an approximation, it did improve the quality of > answers a lot! I did some basic write up of the details plus screenshots > here: Invented non-real world data like this simply doesn't belong in OpenStreetMap I'm afraid. I mean feel free to generate these areas and make them available for geocoding etc but they're not real things and they clearly can't be verified by anybody because they don't actually exist. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
