Re: [Talk-GB] Land Registry postcode tool now running
I've just noticed (changeset 18037116) someone using this Land Registry data search tool to actually populate, rather than simply flag for re-survey postcode data. Given that the Land Registry require a copyright notice, and, unlike OS Open Data, there is no such copyright notice on OSM, is this usage actually legal? I also note that the source is only given as a changeset comment, not a source tag. -- Reply Message -- Nick Allen nick.allen.54 at gmail.com Thu Sep 13 10:29:23 BST 2012 Matt, Thanks for your work on this. I've had a little play it looks ideal for my purpose you should see an improvement in the BR8 area as a direct result. If your not doing it already, could you put somewhere on the front page when the data was updated. My initial plan is to have one session a month correcting adding postcodes, but you may get other views. Regards Nick (Tallguy) Sent from my HTC - Reply message - From: Matt Williams lists at milliams.com To: Talk-GB at openstreetmap.org talk-gb at openstreetmap.org Subject: [Talk-GB] Land Registry postcode tool now running Date: Thu, Sep 13, 2012 01:00 Hi all, During the discussion about the Land Registry 'Price Paid' database discussions I promised that I'm put together a tool to make it more useful. As such I've now got working (to a state I'm happy with) a sub-website on my Postcode Finder to provide an interface to that data. Currently running at http://milliams.dev.openstreetmap.org/postcodefinder/landregistry/ (there's no public link to it yet so you'll have to bookmark it) it does the following: Given a postcode (or postcode fragment) it finds all the houses in the Land Registry database and tries to match each of them up to an address in my Postcode Finder database. If it finds a match it classifies it into one of three classes: - The postcode matches between OSM and the Land Registry - The postcode doesn't match - There's no postcode If you perform a search (like http://milliams.dev.openstreetmap.org/postcodefinder/landregistry/search/?postcode=CV4+8) you see that I've sorted the classes by 'importance' order so wrong postcodes are first, followed by missing postcodes. Then the ones that couldn't be matched to any address and finally the perfect matches. Give it a try yourself. It should be able to handle anything from CV4 level (will take around 10 seconds to load (probably more for big cities like Birmingham and London) down to CV4 8DU. It will almost certainly struggle if you try to put in B or something big like that. If you want to search for all the B1 postcodes but exclude B10, B11 etc (since that would be a very slow query) then just put a space after it in the search box (B1 ) or a plus in the URL (?postcode=B1+). It's probably best to only give it as specific a postcode as is possible (XXN N is quite quick) to keep the load low on the dev server. There's still some things I want to do with it but it's now in a workable state. The data from the OSM database is a few days old now. I'll wait for Geofabrik's ODbL extracts to be released before I update again. It should also be possible for me to extend the service to include and data source which contains house number; street name; postcode so I'll look into that in the future. ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
== Quote == I've just noticed (changeset 18037116) someone using this Land Registry data search tool to actually populate, rather than simply flag for re-survey postcode data. Given that the Land Registry require a copyright notice, and, unlike OS Open Data, there is no such copyright notice on OSM, is this usage actually legal? == End Quote == Hi, Short answer= Yes, its fine, no problems. Long Answer= I was expecting that changeset to be mine, but pleased to see that other people are now using Matt's postcode tool. To answer your question:- The data is available under the standard Open Government Licence (just like the ONS Postcode Centroids that I am also using if it is right above a house), and can therefore be used with attribution. The attribution is on the Contributors page [1] and has been since 31 August 2012. Only a very select few sources are also attributed on the Copyright page [2] and the page makes it clear that you should see the Contributors page for the other sources. Best regards, Rob Nickerson [1] http://wiki.openstreetmap.org/wiki/Contributors#Land_Registry_-_Price_paid_data [2] http://www.openstreetmap.org/copyright ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
Oh I should add that I am a fan of source tags on the objects myself so I add a source:postcode=Land Registry 'Price Paid' data tag (or =ONS Postcode Centroids) to my edits. Not everyone agrees that source tags should be added to the objects, preferring instead to add them to the changeset comment. Regards, Rob ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
Looks like some very good work is going on in Ruislip! When I saw the initial comment I also expected the changeset to be mine as it's the sort of thing I've been doing as well. Although I try to add a source code to everything I add to the data, I'm pretty sure there are times when I miss something. It's quite good to have this double level of history added to any changes - not only can you add a source tag, in my case probably source=survey;Bing;Land Registry but on the occasion where you miss that there is the changeset comment as well which makes it clear. Looks as if this mapper has covered this angle pretty clearly. In a system where most of the 'rules' are governed by what most people have done, it's no surprise there is variation between how we apply a tag showing our sources. Anyway, if you're looking for a project in OSM, adding addresses is easily accomplished. Personally I currently use Keypad-Mapper 3 OSMTracker to do my surveying, and the exercise does me the world of good. If anyone fancies joining in, I've outlined how I carry out the process at http://www.openstreetmap.org/user/Tallguy Regards Nick (Tallguy) On 28/09/13 20:45, Rob Nickerson wrote: == Quote == I've just noticed (changeset 18037116) someone using this Land Registry data search tool to actually populate, rather than simply flag for re-survey postcode data. Given that the Land Registry require a copyright notice, and, unlike OS Open Data, there is no such copyright notice on OSM, is this usage actually legal? == End Quote == Hi, Short answer= Yes, its fine, no problems. Long Answer= I was expecting that changeset to be mine, but pleased to see that other people are now using Matt's postcode tool. To answer your question:- The data is available under the standard Open Government Licence (just like the ONS Postcode Centroids that I am also using if it is right above a house), and can therefore be used with attribution. The attribution is on the Contributors page [1] and has been since 31 August 2012. Only a very select few sources are also attributed on the Copyright page [2] and the page makes it clear that you should see the Contributors page for the other sources. Best regards, Rob Nickerson [1] http://wiki.openstreetmap.org/wiki/Contributors#Land_Registry_-_Price_paid_data [2] http://www.openstreetmap.org/copyright ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 28 September 2013 20:45, Rob Nickerson rob.j.nicker...@gmail.com wrote: == Quote == I've just noticed (changeset 18037116) someone using this Land Registry data search tool to actually populate, rather than simply flag for re-survey postcode data. Given that the Land Registry require a copyright notice, and, unlike OS Open Data, there is no such copyright notice on OSM, is this usage actually legal? == End Quote == Hi, Short answer= Yes, its fine, no problems. Long Answer= I was expecting that changeset to be mine, but pleased to see that other people are now using Matt's postcode tool. To answer your question:- The data is available under the standard Open Government Licence (just like the ONS Postcode Centroids that I am also using if it is right above a house), and can therefore be used with The view has been expressed, recently, on this list that the ONS postcode centroids are still encumbered by the Post Office copyrights. attribution. The attribution is on the Contributors page [1] and has been since 31 August 2012. Only a very select few sources are also attributed on the Copyright page [2] and the page makes it clear that you should see the Contributors page for the other sources. I must admit that I missed that, but I'm not convinced that LR will be happy with such a contorted trail. I think they would like a prominent notice in any place where the data is used. In particular, I can see people creating navigation tools and third party databases without making their uses adequately aware of the sources. Reading the LR FAQ, it is not just about the attribution, but about ensuring that people only make acceptable use of the data, as it is fairly clear that some people have objected to the price component of the data being published. This sort of information isn't being contributed by people actively agreeing to its use by OSM, so you can't really cover it by the and its contributors clause. ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 28 September 2013 23:50, Nick Allen nick.allen...@gmail.com wrote: Anyway, if you're looking for a project in OSM, adding addresses is easily accomplished. Personally I currently use Keypad-Mapper 3 OSMTracker to do my surveying, and the exercise does me the world of good. If anyone fancies joining in, I've outlined how I carry out the process at http://www.openstreetmap.org/user/Tallguy That's how I started back on OSM. However, I am coming to have reservations about the use of GPS for that purpose. I found that I had to re-align everything to Bing for it to be useful, and, as I was mapping front door nodes, rather than buildings, I had to make a second pass to note the layout of the houses and to check for semis versus detached (the latter because of the GPS wander). Although it hasn't come back to the top of my list, I'm considering whether to dispense with the technology for the next tranche and just make notes to allow me to match up with Bing. ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 28 September 2013 20:52, Rob Nickerson rob.j.nicker...@gmail.com wrote: Oh I should add that I am a fan of source tags on the objects myself so I add a source:postcode=Land Registry 'Price Paid' data tag (or =ONS Postcode Centroids) to my edits. Not everyone agrees that source tags should be added to the objects, preferring instead to add them to the changeset comment. I use a mixture of both, depending on the context. Both are flawed, though. Adding to objects doesn't work well when different tags have different sources, and the geometry may have a different one too. It is also very vulnerable to people replacing the whole source with theirs. Adding it to changeset comments means it doesn't survive splits and merges. The database has no knowledge that these actually happened and doesn't record the audit trail necessary to find the true provenance of a node, way or relation. (For this reason, any mechanical redaction is likely to be quite flawed.) This lack of good traceability does worry me, as I see one of the biggest threats to a cloud sourced project like this is people getting over enthusiastic and importing copyright data, possibly in such small individual amounts that no alarms sound, but when aggregated, enough to get a copyright owner angry. My feeling is that the upcoming generation of contributors isn't so steeped in the concept of a map that is untainted by material with restrictive copyrights, so will use the easiest way of getting the data they want added. The most important reason for sources may be to limit what has to be taken down when a take down notice is received. ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
HADW, I can see where your concerns stem from, agree that some form of suitable source code to indicate where the data is derived from could mitigate any potential problems. At present there are some mappers, myself included, who are obtaining postcodes derived from the 'Land Registry price paid data' - we believe we are complying with the current legal requirements are not introducing anything tainted into the database. If we are subsequently shown to be wrong it could help any redaction process if we were all using the same tag to indicate the postcode was derived from the Land Registry Price Paid database. I'm currently planning to start using source:postcode=Land Registry 'price paid' data unless there is any strong guidance to indicate something more suitable. As the postcode is the only part of the data I am using, another simple search would be for any postcodes, and then check their source tag. Regards Nick (Tallguy) On 29/09/13 01:01, OpenStreetmap HADW wrote: On 28 September 2013 20:52, Rob Nickerson rob.j.nicker...@gmail.com wrote: Oh I should add that I am a fan of source tags on the objects myself so I add a source:postcode=Land Registry 'Price Paid' data tag (or =ONS Postcode Centroids) to my edits. Not everyone agrees that source tags should be added to the objects, preferring instead to add them to the changeset comment. I use a mixture of both, depending on the context. Both are flawed, though. Adding to objects doesn't work well when different tags have different sources, and the geometry may have a different one too. It is also very vulnerable to people replacing the whole source with theirs. Adding it to changeset comments means it doesn't survive splits and merges. The database has no knowledge that these actually happened and doesn't record the audit trail necessary to find the true provenance of a node, way or relation. (For this reason, any mechanical redaction is likely to be quite flawed.) This lack of good traceability does worry me, as I see one of the biggest threats to a cloud sourced project like this is people getting over enthusiastic and importing copyright data, possibly in such small individual amounts that no alarms sound, but when aggregated, enough to get a copyright owner angry. My feeling is that the upcoming generation of contributors isn't so steeped in the concept of a map that is untainted by material with restrictive copyrights, so will use the easiest way of getting the data they want added. The most important reason for sources may be to limit what has to be taken down when a take down notice is received. ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 13 September 2012 02:16, Dave F. dave...@madasafish.com wrote: Hi FYI a 6 digit code with no space gives a 'TypeError at /landregistry/search/ 'NoneType' object is unsubscriptable' error. I've now 'fixed' that. Unfortunately, I don't have a way at the moment to allow you to search for postcodes without the space since it's just doing a SELECT in the SQL.I can have a think about how best to manage this though. If I search my code (with a space) it finds it, but if I search by street name it only finds ones State side. Is this a nominatim problem? Could you be more explicit in the steps you took? My postcode finder doesn't use Nominatim inside at all. However, occasionally, when it can't find anything using its internal database, it will offer you the option to do a simple search using Nominatim instead. This is when it says Search for foo in OSM. -- Matt Williams http://milliams.com ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 13 September 2012 10:29, Nick Allen nick.allen...@gmail.com wrote: Matt, Thanks for your work on this. I've had a little play it looks ideal for my purpose you should see an improvement in the BR8 area as a direct result. If your not doing it already, could you put somewhere on the front page when the data was updated. My initial plan is to have one session a month correcting adding postcodes, but you may get other views. Yes, I've been meaning to do this. I'll add that ASAP. -- Matt Williams http://milliams.com ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 13/09/2012 10:38, Matt Williams wrote: Could you be more explicit in the steps you took? Typed postcode clicked Search Under the 'search address' column I clicked [search] Next window said 'Search for 10 Downing Street in OSM. I clicked it. It took me to somewhere in the states on the Nominatim site. Incidentally search for the PM's office (SW1A 2AA) doesn't find any addresses. Dave F. ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
On 13 September 2012 12:08, Dave F. dave...@madasafish.com wrote: On 13/09/2012 10:38, Matt Williams wrote: Could you be more explicit in the steps you took? Typed postcode clicked Search Under the 'search address' column I clicked [search] Next window said 'Search for 10 Downing Street in OSM. I clicked it. It took me to somewhere in the states on the Nominatim site. That sounds like either a Nominatim bug or a lack of data in OSM for Nominatim to find. Incidentally search for the PM's office (SW1A 2AA) doesn't find any addresses. That's because the Land Registry search only shows houses that have sold in the last few months. I will add some text to the page to make this clear. If you search for the postcode in the main search part, you'll find http://milliams.dev.openstreetmap.org/postcodefinder/SW1A%202AA/ -- Matt Williams http://milliams.com ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
[Talk-GB] Land Registry postcode tool now running
Hi all, During the discussion about the Land Registry 'Price Paid' database discussions I promised that I'm put together a tool to make it more useful. As such I've now got working (to a state I'm happy with) a sub-website on my Postcode Finder to provide an interface to that data. Currently running at http://milliams.dev.openstreetmap.org/postcodefinder/landregistry/ (there's no public link to it yet so you'll have to bookmark it) it does the following: Given a postcode (or postcode fragment) it finds all the houses in the Land Registry database and tries to match each of them up to an address in my Postcode Finder database. If it finds a match it classifies it into one of three classes: - The postcode matches between OSM and the Land Registry - The postcode doesn't match - There's no postcode If you perform a search (like http://milliams.dev.openstreetmap.org/postcodefinder/landregistry/search/?postcode=CV4+8) you see that I've sorted the classes by 'importance' order so wrong postcodes are first, followed by missing postcodes. Then the ones that couldn't be matched to any address and finally the perfect matches. Give it a try yourself. It should be able to handle anything from CV4 level (will take around 10 seconds to load (probably more for big cities like Birmingham and London) down to CV4 8DU. It will almost certainly struggle if you try to put in B or something big like that. If you want to search for all the B1 postcodes but exclude B10, B11 etc (since that would be a very slow query) then just put a space after it in the search box (B1 ) or a plus in the URL (?postcode=B1+). It's probably best to only give it as specific a postcode as is possible (XXN N is quite quick) to keep the load low on the dev server. There's still some things I want to do with it but it's now in a workable state. The data from the OSM database is a few days old now. I'll wait for Geofabrik's ODbL extracts to be released before I update again. It should also be possible for me to extend the service to include and data source which contains house number; street name; postcode so I'll look into that in the future. -- Matt Williams http://milliams.com ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Land Registry postcode tool now running
Hi FYI a 6 digit code with no space gives a 'TypeError at /landregistry/search/ 'NoneType' object is unsubscriptable' error. If I search my code (with a space) it finds it, but if I search by street name it only finds ones State side. Is this a nominatim problem? Cheers Dave F. ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
