Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 25/10/17 12:42, Andy Mabbett wrote: wikidata objects often don't say what they are about (besides the name), they are just a bunch of links to wikipedia articles in different languages >>> Once again, please stop making things up. >> I was replying to "A Wikidata tag is just as verifiable as Wikipedia tag" > Whatever you were replying to, the claim you made was - not for the > first time - false, as others have demonstrated. I'm with Martin on this one! As others have demonstrated, there are a number of holes in the way wikidata works which may be fixed over time, but 'just' using a wikidata reference is not 100% reliable. What *IS* nice is the fact that having looked up a wikidata reference, there may be further links to a number of data sources from which the data was downloaded but that may NOT currently include wikipedia pages that are useful to provide background information on object in question. It would be nice to follow a similar pattern when using any third party ID. One currently being worked on in the UK is the 'fhrs' database, and these references are being added to the identified establishments. BUT it would be nice is these were tagged ref:fhrs in the same way as ref:edubase was populated previously, and a link to the reinvent data can be created automatically. ref:wikidata should follow the same pattern where an object is a one to one match for the referenced page, but wikipedia tag should be allowed where the referenced page contains general information about the object ... as in the case of a link to an artist where the object is a sculpture. Until the wikidata id actually identifies the sculpture there should not be a wikidata entry. What is currently missing is a clean set of guidelines for using any third party reference? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 25 October 2017 at 11:33, Martin Koppenhoeferwrote: > 2017-10-25 11:49 GMT+02:00 Andy Mabbett : >> >> On 25 October 2017 at 10:06, Martin Koppenhoefer >> wrote: >> > wikidata objects often don't say what they are about >> > (besides the name), they are just a bunch of links to wikipedia articles >> > in different languages >> >> Once again, please stop making things up. > I was replying to "A Wikidata tag is just as verifiable as Wikipedia tag" Whatever you were replying to, the claim you made was - not for the first time - false, as others have demonstrated. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-10-25 11:49 GMT+02:00 Andy Mabbett: > On 25 October 2017 at 10:06, Martin Koppenhoefer > wrote: > > 2017-10-25 9:27 GMT+02:00 Safwat Halaby : > > > wikidata objects often don't say what they are about > > (besides the name), they are just a bunch of links to wikipedia articles > in > > different languages > > Once again, please stop making things up. > > > I was replying to "A Wikidata tag is just as verifiable as Wikipedia tag", and that's not true, there are significant differences which don't make the two interchangeable. Here's an example for major confusion: https://www.wikidata.org/wiki/Q1368377 (e.g. English and Italian version describe different "things", wikidata describes a "mixture"). there are also https://www.wikidata.org/wiki/Q3734793 https://www.wikidata.org/wiki/Q3734597 and maybe more... Cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Please tell me where the wikipedia link is in e.g. https://www.wikidata.org/wiki/Q37344570 :-) Wikidata does not have to be a bunch of links to wikipedia articles. It has references to 2 external DBs (ODIS & Onroerend erfgoed), so it should be considered notable. I have no idea how many bad items there are in Wikidata, just as I don't know how many bad nodes there are in OSM (e.g. just a name tag). Do we have to throw OSM through the window just because I can find some nodes with just a name tag ? So why do we do this with another project ? m. On Wed, Oct 25, 2017 at 11:06 AM, Martin Koppenhoeferwrote: > 2017-10-25 9:27 GMT+02:00 Safwat Halaby : >> >> >> A Wikidata tag is just as verifiable as Wikipedia tag: Both require >> visiting an external site. Y > > > > no, because wikipedia articles describe what they are about (or get deleted > for lack of substance), wikidata objects often don't say what they are about > (besides the name), they are just a bunch of links to wikipedia articles in > different languages and not too rarely with different content (you have to > decide which linked wikipedia article in which language defines the object). > > Cheers, > Martin > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 25 October 2017 at 10:06, Martin Koppenhoeferwrote: > 2017-10-25 9:27 GMT+02:00 Safwat Halaby : > wikidata objects often don't say what they are about > (besides the name), they are just a bunch of links to wikipedia articles in > different languages Once again, please stop making things up. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-10-25 11:06 GMT+02:00 Martin Koppenhoefer: > 2017-10-25 9:27 GMT+02:00 Safwat Halaby : > >> >> A Wikidata tag is just as verifiable as Wikipedia tag: Both require >> visiting an external site. Y > > > > no, because wikipedia articles describe what they are about (or get > deleted for lack of substance), wikidata objects often don't say what they > are about (besides the name), they are just a bunch of links to wikipedia > articles in different languages and not too rarely with different content > (you have to decide which linked wikipedia article in which language > defines the object). > > Nice argument! https://xkcd.com/285/ Example https://tools.wmflabs.org/reasonator/?=60 > Cheers, > Martin > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-10-25 9:27 GMT+02:00 Safwat Halaby: > > A Wikidata tag is just as verifiable as Wikipedia tag: Both require > visiting an external site. Y no, because wikipedia articles describe what they are about (or get deleted for lack of substance), wikidata objects often don't say what they are about (besides the name), they are just a bunch of links to wikipedia articles in different languages and not too rarely with different content (you have to decide which linked wikipedia article in which language defines the object). Cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
>Wow, i had to read this twice to really believe what i was reading >here. >Seems you are still in deep denial about the fundamental differences >between OSM and Wikipedia. Christopher, A Wikidata tag is just as verifiable as Wikipedia tag: Both require visiting an external site. Yuri made no further claims about anything fundamental. Yuri's second argument is that Wikidata tags are more stable, which is objectively true. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> You have advanced to justification "You have advanced no justification..." On 3 October 2017 at 13:43, Andy Mabbettwrote: > On 2 October 2017 at 19:36, Frederik Ramm wrote: > >> Instead, I have Yuri adding, modifying, >> and re-adding Wikidata tags all over the planet and ignoring most calls >> for moderation, and Andy Mabbett from England editing supermarkets in >> Germany. > > I've asked you once already to drop your nationalist comments; why are > you repeating them? > > You have advanced to justification for suggesting that someone who > happens to live in England should not edit items in Germany, nor > indeed vice versa, nor for someone whose home and edits are in any two > different countries. > > -- > Andy Mabbett > @pigsonthewing > http://pigsonthewing.org.uk -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 2 October 2017 at 19:36, Frederik Rammwrote: > Instead, I have Yuri adding, modifying, > and re-adding Wikidata tags all over the planet and ignoring most calls > for moderation, and Andy Mabbett from England editing supermarkets in > Germany. I've asked you once already to drop your nationalist comments; why are you repeating them? You have advanced to justification for suggesting that someone who happens to live in England should not edit items in Germany, nor indeed vice versa, nor for someone whose home and edits are in any two different countries. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-10-03 8:11 GMT+02:00 Yuri Astrakhan: > Martin, while it is fascinating to learn about Aldi, its history, and > possible ways to organize information about it, isn't it a moot point for > our discussion? > I just took it as an example because I think it works to illustrate several problems (I didn't bring it up myself, I am not affiliated with them in any way of course). One of these is wikipedia tags perfectly fitting for an object in OSM and related wikidata not fitting, but being misleading and factually wrong (almost everything on the wikidata object was wrong initially, and keeps being wrong and got even worse after a lot of modification following our discussion here). Seems strange, as the wikidata object was initially created for the article, but it is like this. It could be seen as alarming if not even proficient WD editors are able to solve the issues for an object in the spotlight, in what condition will the rest of it be, but I think this would be unfair, because it is clear you can't reasonably make good edits on a topic from a different cultural context you are not familiar with, where the information is mainly available in a language you don't speak, and from a field you are likely not very interested in (retail). > We are talking about Wikipedia, and how we link to it. There is only one > Aldi Wikipedia article that can be connected to: > actually we are talking about wikidata tags being added automatically, without human verification, based on wikipedia tags that are present. The fact that you can link only one Wikipedia article to a Wikidata object could be a design problem, but I am not completely sure > > This is the current behavior of the iD editor: you type in Wikipedia page, > and it automatically updates Wikidata ID, storing both values. If you > think this is incorrect, please start a discussion, > yes, I do believe it is the same or a similar problem with iD behaving like this, and the discussion should have been started by the iD developers before they added this feature (because it is covered by the automated edits guidelines, or if it's not currently, it should be). > But this has been the automatic software behavior for a long time. Most iD > users would not even know that they have updated Wikidata tag, > it's complicated, because most iD users generally don't know about tags and them being set. They don't even see them per default (AFAIK), its a pro-feature in iD to see all tags (just verified, and "all tags" is there, but you have to unfold it). I'm not happy with this, but there are also arguments that it is beneficial for a certain group of mappers (less things to learn, easier to start). > so lets not treat "wikidata" as some magical unicorn that links to > something bigger and better - it is simply a link to Wikipedia. > its not true, you keep repeating this, but WD has its own semantic tags that describe what it is about, the objects don't claim to be representing an article, they claim to be representing a real world thing, and have links to WP articles that are supposed to be about this same thing. Wikipedia itselft says about wikidata: "Wikidata is a collaboratively edited knowledge base operated by the Wikimedia Foundation. It is intended to provide a common source of data which can be used by Wikimedia projects such as Wikipedia, and by anyone else, under a public domain licence. This is similar to the way Wikimedia Commons provides storage for media files and access to those files for all Wikimedia projects, and which are also freely available for reuse." Cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Hi Martin, Am 2017-10-03 um 00:28 schrieb Martin Koppenhoefer: > indeed it’s not helping the quality if editors are not familiar with the > language specifics for the area of the things they edit (this is true for all > UGC, be it osm, wikidata, etc). Aldi Sud does not make sense, it’s either Süd > or, if you really have to (e.g. domain names), Sued. > https://www.wikidata.org/wiki/Q41171672 > > This kind of fiddling leads to objects like this: > https://www.wikidata.org/wiki/Q125054 > inception 1913 > founded by Karl and Theo Albrecht, born 1920 and 1922. > Founded 7/9 years before their birth? > > It is also not true that aldi nord and süd result or follow from the > splitting of Aldi, they result from the split of Albrecht KG. Not even the > founding year 1960 for the parts is correct, it’s 1961 (according to wp and > company website) > > It also still claims Aldi is a GmbH & Co. KG and even has 1 reference for > this (german wikipedia), while the German wikipedia actually has a long > paragraph trying to explain the structure and saying there are 66 different > regional GmbH & Co. KG, plus other companies like the ALDI Einkauf GmbH & Co. > oHG or the ALDI SÜD Dienstleistungs-GmbH & Co. oHG, i.e. it’s a group of > companies, a concern. > Here’s a list of parts of Aldi Süd: > https://unternehmen.aldi-sued.de/de/impressum/ > > It can all be fixed of course, but I’m curious how all these errors have > gotten there. There’s still more wrong than correct in this object. The wrong name (Sud instead of Sued or Süd) was added by user Pingsonthewing who probably does not live in Germany or any other German speaking country but tries to actively push Wikidata into OSM. https://www.wikidata.org/w/index.php?title=Q41171672=567343632 Best regards Michael -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) signature.asc Description: OpenPGP digital signature ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Martin, while it is fascinating to learn about Aldi, its history, and possible ways to organize information about it, isn't it a moot point for our discussion? We are talking about Wikipedia, and how we link to it. There is only one Aldi Wikipedia article that can be connected to: * German https://www.wikidata.org/wiki/Special:GoToLinkedPage?itemid= Q125054=dewiki * English https://www.wikidata.org/wiki/Special:GoToLinkedPage?itemid= Q125054=enwiki This is the current behavior of the iD editor: you type in Wikipedia page, and it automatically updates Wikidata ID, storing both values. If you think this is incorrect, please start a discussion, and we may want to change that. But this has been the automatic software behavior for a long time. Most iD users would not even know that they have updated Wikidata tag, so lets not treat "wikidata" as some magical unicorn that links to something bigger and better - it is simply a link to Wikipedia. It is really up to the software to generate a proper link to Wikipedia. It could be generated just like I showed above, or by transforming wikipedia tag, hoping that the page is still the same. In either case, you only get a link. On Mon, Oct 2, 2017 at 6:28 PM, Martin Koppenhoeferwrote: > > > sent from a phone > > On 2. Oct 2017, at 20:36, Frederik Ramm wrote: > > and Andy Mabbett from England editing supermarkets in > Germany. > > > > indeed it’s not helping the quality if editors are not familiar with the > language specifics for the area of the things they edit (this is true for > all UGC, be it osm, wikidata, etc). Aldi Sud does not make sense, it’s > either Süd or, if you really have to (e.g. domain names), Sued. > https://www.wikidata.org/wiki/Q41171672 > > This kind of fiddling leads to objects like this: https://www.wikidata. > org/wiki/Q125054 > inception 1913 > founded by Karl and Theo Albrecht, born 1920 and 1922. > Founded 7/9 years before their birth? > > It is also not true that aldi nord and süd result or follow from the > splitting of Aldi, they result from the split of Albrecht KG. Not even the > founding year 1960 for the parts is correct, it’s 1961 (according to wp and > company website) > > It also still claims Aldi is a GmbH & Co. KG and even has 1 reference for > this (german wikipedia), while the German wikipedia actually has a long > paragraph trying to explain the structure and saying there are 66 different > regional GmbH & Co. KG, plus other companies like the > ALDI Einkauf GmbH & Co. oHG or the *ALDI SÜD > Dienstleistungs-GmbH & Co. oHG*, i.e. it’s a group of companies, a > concern. > Here’s a list of parts of Aldi Süd: > https://unternehmen.aldi-sued.de/de/impressum/ > > It can all be fixed of course, but I’m curious how all these errors have > gotten there. There’s still more wrong than correct in this object. > > cheers, > Martin > > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
sent from a phone > On 2. Oct 2017, at 20:36, Frederik Rammwrote: > > and Andy Mabbett from England editing supermarkets in > Germany. indeed it’s not helping the quality if editors are not familiar with the language specifics for the area of the things they edit (this is true for all UGC, be it osm, wikidata, etc). Aldi Sud does not make sense, it’s either Süd or, if you really have to (e.g. domain names), Sued. https://www.wikidata.org/wiki/Q41171672 This kind of fiddling leads to objects like this: https://www.wikidata.org/wiki/Q125054 inception 1913 founded by Karl and Theo Albrecht, born 1920 and 1922. Founded 7/9 years before their birth? It is also not true that aldi nord and süd result or follow from the splitting of Aldi, they result from the split of Albrecht KG. Not even the founding year 1960 for the parts is correct, it’s 1961 (according to wp and company website) It also still claims Aldi is a GmbH & Co. KG and even has 1 reference for this (german wikipedia), while the German wikipedia actually has a long paragraph trying to explain the structure and saying there are 66 different regional GmbH & Co. KG, plus other companies like the ALDI Einkauf GmbH & Co. oHG or the ALDI SÜD Dienstleistungs-GmbH & Co. oHG, i.e. it’s a group of companies, a concern. Here’s a list of parts of Aldi Süd: https://unternehmen.aldi-sued.de/de/impressum/ It can all be fixed of course, but I’m curious how all these errors have gotten there. There’s still more wrong than correct in this object. cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
>And for further clarification, many Wikidata tags that have been added in the past six months have been added in blatant violation of the mechanical edit guidelines, and we should think about whether they should be removed again until such time as people actually have the time to do it properly and manually. I would support such a move. Cheerio John On 2 October 2017 at 14:39, Frederik Rammwrote: > Hi, > > On 02.10.2017 13:06, Christoph Hormann wrote: > > For clarification - since this is often a point of misunderstandings - > > if an edit is a mechanical edit or import does not depends on the tools > > used, it depends on if the modifications are made individually feature > > by feature based on individual, informed assessment of the local > > situation or if it is made in bulk without each feature being evaluated > > individually - and doing this by clicking a button a hundred times > > instead of scripting it of course does not qualify as manual > > evaluation. > > And for further clarification, many Wikidata tags that have been added > in the past six months have been added in blatant violation of the > mechanical edit guidelines, and we should think about whether they > should be removed again until such time as people actually have the time > to do it properly and manually. > > Bye > Frederik > > -- > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Hi, On 02.10.2017 13:06, Christoph Hormann wrote: > For clarification - since this is often a point of misunderstandings - > if an edit is a mechanical edit or import does not depends on the tools > used, it depends on if the modifications are made individually feature > by feature based on individual, informed assessment of the local > situation or if it is made in bulk without each feature being evaluated > individually - and doing this by clicking a button a hundred times > instead of scripting it of course does not qualify as manual > evaluation. And for further clarification, many Wikidata tags that have been added in the past six months have been added in blatant violation of the mechanical edit guidelines, and we should think about whether they should be removed again until such time as people actually have the time to do it properly and manually. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Hi, On 01.10.2017 13:13, Christoph Hormann wrote: > I find it fascinating how both you and Yuri seem to be eager to always > deflect the discussion from the main subject, namely the automated > editing/addition of wikidata IDs and misinterpreting constructive > critique of that as an attempt to tell local mappers what tags they may > and may not add to the things they map. I would have much less of an issue with the wikidata stuff if it was indeed added by local mappers. Instead, I have Yuri adding, modifying, and re-adding Wikidata tags all over the planet and ignoring most calls for moderation, and Andy Mabbett from England editing supermarkets in Germany. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Monday 02 October 2017, Stefan Keller wrote: > > I would like to auto-add all the corresponding wikidata based on > > wikipedia, for all remaining objects, using JOSM's "Fetch Wikidata > > IDs". > > Pls. correct me if I'm missing something here. Though "auto-add" is > perhaps not the best notion, it's still a human in control of JOSM. > To me, OSM really should remain a welcoming, inclusive do-ocracy. > So, let's look forward. For clarification - since this is often a point of misunderstandings - if an edit is a mechanical edit or import does not depends on the tools used, it depends on if the modifications are made individually feature by feature based on individual, informed assessment of the local situation or if it is made in bulk without each feature being evaluated individually - and doing this by clicking a button a hundred times instead of scripting it of course does not qualify as manual evaluation. Manual evaluation would of course be pointless if the data is not verifiable which is why so much discussion on the matter evolved about the problem of verifiability. If the data is not verifiable you should neither add it manually nor through automated edits. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Hi Yuri and all Just my 2 cents: I think the reasoning of Lester makes sense: be careful about semi-automatic adding wikidata tags to any OSM object with a Wikipedia tag. Applied to a subset of OSM this already took place and now there are some objects remaining, which will need to be curated by hand. Specifically we're speaking about 750 remaining OSM objects with wrong Wikidata tags (disambig. instead of "real" resources). And that's a feasible number to be done by hand, isn't it? Just for the ground truth following input: * Wikipedia tags are used in several services, apps and tools, like Nominatim(!), OSMNames.org, historic.places, etc.. * Wikidata tags are used by Mapbox, OpenMapTiles, Mapzen(?), etc. So both tags have already their place. In this thread it's Yuri just asked this initially: > I would like to auto-add all the corresponding wikidata based on wikipedia, > for all remaining objects, using JOSM's "Fetch Wikidata IDs". Pls. correct me if I'm missing something here. Though "auto-add" is perhaps not the best notion, it's still a human in control of JOSM. To me, OSM really should remain a welcoming, inclusive do-ocracy. So, let's look forward. :Stefan P.S. To anybody eager to debate, IMHO the following OSM Wiki pages need some care, and this is something really useful to discuss and update: * https://wiki.openstreetmap.org/wiki/Key:wikipedia * https://wiki.openstreetmap.org/wiki/Key:wikidata 2017-10-02 10:22 GMT+02:00 Lester Caine: > On 19/09/17 21:03, Yuri Astrakhan wrote: >> There is now a relatively small number of OSM nodes and relations >> remaining, that have wikipedia, but do not have wikidata tags. iD editor >> already automatically adds wikidata to all new edits, so finishing up >> the rest automatically seems like a good thing to do, as that will allow >> many new quality control queries. I would like to auto-add all the >> corresponding wikidata based on wikipedia, for all remaining objects, >> using JOSM's "Fetch Wikidata IDs". > > Yuri > I'm going to wind things back in a bit here as the discussion seems to > be widely disjointed. > > I think there needs to be a formal discussion as to just what secondary > data sources are preferred when adding additonal data to OSM objects. > Wikipedia is a useful secondary source for a vast range of material, but > some objects in OSM will not be 'notable' enough for wikipedia to allow > an article to exist, so an alternative mechanism is needed for those > objects. wikidata may well be a suitable alternative, but the simple > fact that a more detailed article for a wikidata object by not be > accepted makes wikidata something of a problem as well. While > wikipedia/wikidata provide a sort of standard framework for additional > data, the primary link from an OSM object should perhaps be to websites > specific to the object rather than the filtered wikipedia view of the > world? > > Some of the tangential debate has been re ADDING links to OSM object > that have not yet been tagged with something suitable, and this is where > cross matching these items depends on the information already available > on in the OSM tags. I have still to be convinced that wikidata is > 'independent' enough to be a reliable source of cross-reference links, > especially where other reference tags are already used. UK Schools we > have added the reference from the schools database. I've not looked to > see if a wikidata 'view' of that data is available, but I would not look > to add wikidata id's in addition to the school id's - BUT the wikipedia > reference has been added where the schools are notable enough to warrant > an article. I WOULD only look to tidying these objects to ADD the > wikidata id if one was also checking that all three elements are > correct, rather than simply automatically adding them. Whilst I was > processing that data during the UK group push on it there were often a > lot of corrections made to get the right 'set' of data on a school > object. While only a small number of objects were actually wrong, that > is enough to justify needing a manual cross check of some sort. > > -- > Lester Caine - G8HFL > - > Contact - http://lsces.co.uk/wiki/?page=contact > L.S.Caine Electronic Services - http://lsces.co.uk > EnquirySolve - http://enquirysolve.com/ > Model Engineers Digital Workshop - http://medw.co.uk > Rainbow Digital Media - http://rainbowdigitalmedia.co.uk > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 19/09/17 21:03, Yuri Astrakhan wrote: > There is now a relatively small number of OSM nodes and relations > remaining, that have wikipedia, but do not have wikidata tags. iD editor > already automatically adds wikidata to all new edits, so finishing up > the rest automatically seems like a good thing to do, as that will allow > many new quality control queries. I would like to auto-add all the > corresponding wikidata based on wikipedia, for all remaining objects, > using JOSM's "Fetch Wikidata IDs". Yuri I'm going to wind things back in a bit here as the discussion seems to be widely disjointed. I think there needs to be a formal discussion as to just what secondary data sources are preferred when adding additonal data to OSM objects. Wikipedia is a useful secondary source for a vast range of material, but some objects in OSM will not be 'notable' enough for wikipedia to allow an article to exist, so an alternative mechanism is needed for those objects. wikidata may well be a suitable alternative, but the simple fact that a more detailed article for a wikidata object by not be accepted makes wikidata something of a problem as well. While wikipedia/wikidata provide a sort of standard framework for additional data, the primary link from an OSM object should perhaps be to websites specific to the object rather than the filtered wikipedia view of the world? Some of the tangential debate has been re ADDING links to OSM object that have not yet been tagged with something suitable, and this is where cross matching these items depends on the information already available on in the OSM tags. I have still to be convinced that wikidata is 'independent' enough to be a reliable source of cross-reference links, especially where other reference tags are already used. UK Schools we have added the reference from the schools database. I've not looked to see if a wikidata 'view' of that data is available, but I would not look to add wikidata id's in addition to the school id's - BUT the wikipedia reference has been added where the schools are notable enough to warrant an article. I WOULD only look to tidying these objects to ADD the wikidata id if one was also checking that all three elements are correct, rather than simply automatically adding them. Whilst I was processing that data during the UK group push on it there were often a lot of corrections made to get the right 'set' of data on a school object. While only a small number of objects were actually wrong, that is enough to justify needing a manual cross check of some sort. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> Tomas, this is what I understand from what you are saying: > * You download a geotagging wikidata dump and generate a table with > latitude, longitude, and a wiki page title. > * You also generate the same table from OSM for all nodes, ways (using geo > centroid?), and relations (using ??) > * you compare article titles between the two, and when OSM has something > that Wikipedia doesn't, you search automatically by geo proximity, or you > let users fix it or ?? Relations (abstract colletions, not multipoligons as such) and long ways, such as rivers, are ignored. There are different mechanisms to sort those out. Found problems are placed on a list of problems which is then reviewed by users, research is done and when possible - problems are fixed on problem side (osm or wiki). > If I understood you correctly (and please correct my understanding if I did > not), it wouldn't work for the whole planet, simply because the average > distance between what OSM has and what Wikidata has is far too great to be > useful. If coordinates a too far apart it is reported as an error and has to be fixed. Usually this is the case of incorrect coordinates in wikipedia because of copying of other article with coordinates (say for the similar object like hillfort or lake) and forgetting to update the coordinates. There were cases when objects in Lithuania had coordinates in Africa :-) And such cases were identified with the same success as "closer" mis-matches. It is not important if distance is 5km or 5000km. The approximation we use is something like 1km which is way smaller than Lithuania :-) So I do not see why this mechanism would not work globally. > current state of the world OSM data is that there are only 17% of nodes are > within 10 meters of their Wikidata counterpart. It is not important for a coordinate to be exactly the same. For example if you have a coordinate for a lake or even hillfort, any coordinate within a radius of hundred meters (for a hillfort) or even more (for a lake) is perfectly ok. You can distinguish by wikipedia data what type of object that is: waterbody or something else. So it is possible to adjust the proximity setting for specific object type. > If we count ways and > relations, it drops to 11% -- http://tinyurl.com/ybp4tp7a This is what we've seen in the beginning before starting to fix the data. > In other words, with your approach, you can detect when OSM's wikipedia tag > is no longer correct, because Wikipedia geo dump no longer has it. But > afterwards you have to go and fix it by hand. And this is pretty much the > only operation you can do with this approach. You cannot analyze tens of > thousands of existing wikipedia tags that are pointing to links, disambigs, > people, tree species, places of business - you can simply mark them as "geo > missing in Wikipedia". Identifying them as "missing in wikipedia" proved to be enough. > I took a quick look at the various quality control queries I built on the > cleanup page. Lithuania does seem pretty clean, with only one > disambiguation at the moment (has been there for 4 months) - > https://www.openstreetmap.org/node/1717783246 - but both have the same > location, two airports that point to a list - > https://www.openstreetmap.org/node/1042034645 and > https://www.openstreetmap.org/node/1042034660 . None of these issues are > possible to find with your approach, or detect renaming. For the rest of the > world, the situation is much worse. All three are successfully identified in a large (435 item) problem list of "objects with wikipedia tag where wikipedia article does not have coordinates or coordinates a too far apart". -- Tomas ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> > > I will repeat that this is not something which COULD be done, this > comparison is something, what IS ACTUALLY DONE and has been done for > years. Tomas, this is what I understand from what you are saying: * You download a geotagging wikidata dump and generate a table with latitude, longitude, and a wiki page title. * You also generate the same table from OSM for all nodes, ways (using geo centroid?), and relations (using ??) * you compare article titles between the two, and when OSM has something that Wikipedia doesn't, you search automatically by geo proximity, or you let users fix it or ?? If I understood you correctly (and please correct my understanding if I did not), it wouldn't work for the whole planet, simply because the average distance between what OSM has and what Wikidata has is far too great to be useful. Maybe Lithuania, being a relatively small area with a very active community has been kept up in a perfect form (and each geo point is identical in both Wikidata & OSM, which might be a licensing issue), but the current state of the world OSM data is that there are only 17% of nodes are within 10 meters of their Wikidata counterpart. If we count ways and relations, it drops to 11% -- http://tinyurl.com/ybp4tp7a In other words, with your approach, you can detect when OSM's wikipedia tag is no longer correct, because Wikipedia geo dump no longer has it. But afterwards you have to go and fix it by hand. And this is pretty much the only operation you can do with this approach. You cannot analyze tens of thousands of existing wikipedia tags that are pointing to links, disambigs, people, tree species, places of business - you can simply mark them as "geo missing in Wikipedia". I took a quick look at the various quality control queries I built on the cleanup page. Lithuania does seem pretty clean, with only one disambiguation at the moment (has been there for 4 months) - https://www.openstreetmap.org/node/1717783246 - but both have the same location, two airports that point to a list - https://www.openstreetmap.org/node/1042034645 and https://www.openstreetmap.org/node/1042034660 . None of these issues are possible to find with your approach, or detect renaming. For the rest of the world, the situation is much worse. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
>> So you have two tables of same structure. Voila. You can compare >> anything (title, coordinates), in any direction with some >> approximation if needed etc. No OSM wikidata involved at all. > > Thomas, this will not work. Matching wikidata & osm by coordinates is > useless, because the coordinates differ too much -- see the hard data proof > <...> I will repeat that this is not something which COULD be done, this comparison is something, what IS ACTUALLY DONE and has been done for years. -- Tomas ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
You can compareanything (title, coordinates), in any direction with someapproximation if needed etc. That's the root of an evil. That comparison have to be done manually. I really don't understand why wikidata needs to be added. It needs to be added to avoid manual fixes of wikipedia links,because wikipedia's articles names aren't constant. That looks like inventing sphisticated comparison procedurewhich has to be done manualy only not to have wikidata tags. What's the point in not having wikidata? 02.10.2017, 05:15, "john whelan":Rather than fill OSM up with automated edits that have not even been discussed with the local community can we think more about functionality? Since an OSM object has lat and long value and it appears that wiki whatever also has one the entries can be linked. "This gives you a very simple table with: lat/lonThat/page_title. No parsing or anything else involved. You then take data from OSM - lat/lon/wikipedia.tag So you have two tables of same structure. Voila. You can compareanything (title, coordinates), in any direction with someapproximation if needed etc. No OSM wikidata involved at all." I really don't understand why wikidata needs to be added. Note the word need, I'm missing the requirement somehow that overides following normal OSM practices. Cheerio John On 1 Oct 2017 7:19 pm, wrote:Hi everybody.We already accepted wikipedia links keep in mind that wiki article isn't the same abstraction as OSM object.And the way we make a reference on wiki articles differs over time.It was a link, it was an article name, it was a name with language prefix.Wikidata id is a way to make a refererence onto wikipedia articlewhich wiki community states to be the right and consistent one.Why don't we just accept the recomended way of referencing wiki articles?That doesn't make wikipedia articles more or less verifable on the ground.But that makes the link persistent agains the changes in wikipedia.01.10.2017, 16:38, "Christoph Hormann" :> On Sunday 01 October 2017, Marc Gemis wrote:>> If I noticed an inception date on a information sign next to a>> building, is this original research or a secondary source ?>> The date on the sign is verifiable as a signed date, not necessarily as> a date connected to the building. Historic information like dates from> before the timeframe witnessed by today's mappers is something that is> problematic in OSM in general.>> As any historian will be able to tell you history as a science is not> about analyzing events of the past in the same way you analyze present> day observations in empiric science disciplines but about analyzing and> interpreting sources - in this way it resembles the approach of> Wikipedia more than that of OSM.>>> If a Wikidata contributor/Wikipedian researches websites like Orbis,>> Onroerend erfgoed, etc and finds different inception dates and>> records them all, is it bad that s/he uses secondary sources instead>> of the information sign on the ground ?>> I don't want to judge the different approaches of Wikipedia and OSM -> both have their pros and cons. As a contributor i am more comfortable> with the OSM approach but i would never say that collecting information> from secondary sources is inherently less valuable than collecting> empiric data.>>> I wonder whether OSM is really always based on original research, or>> whether we sometimes just want to believe this.>> Well - human perception is of course always connected to our past> experience and beliefs, we cannot observe our environment in a truly> neutral way. And of course with armchair mapping much of the data in> OSM is produced by people who have never been near the place they map> based on the belief that what the image layer they use depicts what is> actually there (which is sometimes not the case - like with the iconic> demolished buildings) and that what people think they see in the images> actually resembles the impression they would have on the ground (which> is also frequently not the case - prominent example would be here:> http://www.openstreetmap.org/relation/7142214). But the key is every> information in OSM has to be able to stand up to scrutiny by local> mappers - a local mapper standing in front of the object needs to be> able to tell if it is correct or not - even if different mappers could> in borderline cases come to different assessments on the matter.>> --> Christoph Hormann> http://www.imagico.de/>> ___> talk mailing list> talk@openstreetmap.org> https://lists.openstreetmap.org/listinfo/talk___talk mailing listtalk@openstreetmap.orghttps://lists.openstreetmap.org/listinfo/talk___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sun, Oct 1, 2017 at 3:45 PM, Tomas Straupiswrote: > > Tomas, you claimed that "It adds NO value." This is demonstrably wrong. > You > > are right that the same fixing was done for years. But until wikidata > tag, > > there was no easy way to FIND them. > > There always was. > You simply take wikipedia provided geo-tags dump like > https://dumps.wikimedia.org/ltwiki/latest/ltwiki-latest-geo_tags.sql.gz > > This gives you a very simple table with: lat/lon/page_title. > No parsing or anything else involved. > You then take data from OSM - lat/lon/wikipedia.tag > So you have two tables of same structure. Voila. You can compare > anything (title, coordinates), in any direction with some > approximation if needed etc. No OSM wikidata involved at all. > Thomas, this will not work. Matching wikidata & osm by coordinates is useless, because the coordinates differ too much -- see the hard data proof in the prev email. The only way you can make any useful calculation is if you analyze the entirety of Wikidata graph, and merge it with OSM objects, and expose it to other users so that they can figure out what is right or broken. That's exactly what my Wikidata+OSM service allows users to do. > If wikipedia page moves - title is gone from this dump and the new > one appears on the same coordinates. You can map them very quickly. > Theoretically you can update OSM data automatically, but usually if > wikipedia title has changed, it means that something has changed in > the object on the ground, so maybe something else has to be changed in > OSM data as well (for example name). > Again - not possible - because coordinate matching is mostly useless. Also, no, usually wikipedia titles change not because something changed on the ground, but because of a conflict with a similarly named place somewhere else. People usually rename the original page to a more specific name, and create a new page in its place listing all the disambiguations. This is what breaks titles most often. We now have about 800 left (after thousands already fixed), plus potentially thousands more of those that have not been tagged with wikidata tag yet. > > I'm just saying the same could be done without wikidata tags. > As explained by me in one of the first emails, and by Andy, and a few others, it cannot be done **as easily**. You can build a complex system if you have enough disk space (~1TB), and do a local resolve of wikipedia -> wikidata, and build a complex service on top of it. Or you can simply add a single tag that has already been added to 90% of cases, and use off-the-shelve query engine to merge the data, and let everyone use it. > > See above. What are practical advantages of your method? > Because theoretically you are taking a set A, creating a new set B > from this A, and then you're trying to fix A according to B. This is > logical nonsense :-) There is no point of putting this B into OSM. > This is a temporary data which could be stored in your local "error > checking" database. > Strawman argument :) For each object that has a tag, I use JOSM to get corresponding wikidata tag, and upload that data to OSM. The moment it is uploaded, other systems, such as my wikidata+osm service, get that data. Then community, without my involvement, can analyze the data with many different queries, and fix all the errors they find. If I haven't uploaded the data to OSM, only I would be able to see it, and only I would be able to fix it. I don't know all the different ways community may query the data (I'm already getting hundreds of thousands of queries). Its a tool that helps community. > > 550 objects globally... Well... :-) You should see from here, that > the problem is finding people who want to FIX, not finding problems... > 750 is number NOW. It used to be many thousands. And was all fixed, by volunteers. For just the most obvious of queries. There are many more fixes that needs to happen - see wikipedia link cleanup project on osm wiki. So once the problems are identified, they get solved. Finding them is the problem. > I'm arguing against idea that wikipedia tag is outdated or in any way > worse. But this is exactly what I have been showing with my data about broken tags. Do you have any data to say that it is not worse? > Yes, OSM would not be born > without a geek idea, but it would not have reached what it is now if > it would not be easy to understand for non geeks. Wikidata tag is > totally non-understandable to non-geeks. > Wikidata does not need to be understood by geeks or non-geeks. It's an ID, and everyone understands that concept, and most people don't touch tags they don't understand. Just like mapillary ID, or tons of other local government IDs. The tools we have, like iD editor, can easily work with these IDs without non-geeks as you call them understanding it. The query system also doesn't need to be understood to be used - you simply share the link
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 10/1/2017 5:39 PM, Yuri Astrakhan wrote: Lastly, if the coordinates are different, you may not copy it from OSM to Wikidata because of the difference in the license. Just for clarity and anyone reading the archives later, copying from Wikidata to OSM is also a problem because Wikidata permits coordinate sources like Wikipedia or Google Earth. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
John, I guess it is always good to talk as a data scientist - with numbers and facts. Here's why matching by coordinates would not work. This query calculates the distance between the OSM nodes, and the coordinates that Wikidata has for those nodes. I only looked at nodes, because ways and relations are even more incorrect - Wiki only has a center point. The results are bucketed by the distance (in km) - the bigger the distance, the bigger the mismatch between OSM and Wikipedia. As you can see, only a small number of nodes are accurate to 10 meters.Query: http://tinyurl.com/ybp4tp7a diff in km number of nodes <0.01 75,027 <0.1 131,644 <0.5 147,637 <1 46,891 <2 28,049 <5 10,792 <10 3,537 10+ 7,239 Is this a convincing argument why we should have a Wikipedia/Wikidata link, as oppose to calculate it? The other issue is why we need Wikidata links - while I have said it many times, let me say it again. Because the current system is badly broken - as is evident by tens of thousands of errors that my approach has uncovered. I am not advocating to delete Wikipedia tag. Only that when you use wikipedia tag, it creates a burden on the community to maintain, and community is clearly unable to keep up with the changes on the Wikipedia side. So instead of using just the bad link (page title), I am advocating to use a good link (wikidata). We are already using it for 90%. Why not fill in the last 10%? It does not change anything of how you do your mapping. It simply helps those who want to fix errors, or view corresponding wikipedia articles even if it gets renamed. On Sun, Oct 1, 2017 at 8:50 PM, john whelanwrote: > >Assuming my above arguments has convinced you > > No I still do not see a requirement here, but there again I'm only part of > the community and that's the concern you appear to be ramming this down our > threats. As for what iD does or does not do, I don't see that is relevant. > > Why does OSM need it and why are you unable to put forth a convincing > argument that is accepted by the community? A ninety percent acceptance > rate will be fine but I'm not seeing it. > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
>Assuming my above arguments has convinced you No I still do not see a requirement here, but there again I'm only part of the community and that's the concern you appear to be ramming this down our threats. As for what iD does or does not do, I don't see that is relevant. Why does OSM need it and why are you unable to put forth a convincing argument that is accepted by the community? A ninety percent acceptance rate will be fine but I'm not seeing it. Cheerio John On 1 October 2017 at 20:39, Yuri Astrakhanwrote: > On Sun, Oct 1, 2017 at 8:15 PM, john whelan wrote: > >> Since an OSM object has lat and long value and it appears that wiki >> whatever also has one the entries can be linked. >> > > Not so. The data is very often different between wikipedia, wikidata, and > OSM. Also, the same location could be a square, a famous sculpture within > that square, and some commemorative plaque on it, and all could have some > wikipedia/wikidata entry. Matching them up requires humans, and cannot > reliably be done by an algorithm in a large number of cases. Lastly, if the > coordinates are different, you may not copy it from OSM to Wikidata because > of the difference in the license. > >> >> "This gives you a very simple table with: lat/lon/page_title. >> No parsing or anything else involved. >> You then take data from OSM - lat/lon/wikipedia.tag >> So you have two tables of same structure. Voila. You can compare >> anything (title, coordinates), in any direction with some >> approximation if needed etc. No OSM wikidata involved at all." >> > > See above, this cannot be done with any reasonable reliability by > automatic means. You will end up with an incredible amount of unreliable > data. Feel free to discuss deleting of both Wikipedia and Wikidata tags, > but I seriously doubt the community will go for it. > >> >> I really don't understand why wikidata needs to be added. Note the word >> need, I'm missing the requirement somehow that overides following normal >> OSM practices. >> > > Assuming my above arguments has convinced you -- that we must manually > determine the match between an OSM feature and a Wikipedia article, lets > discuss how best to link to Wikipedia. There are two options: link by > article title, and link by Wikidata ID. The first one causes many errors - > because titles get renamed, and old titles are reused for other meanings. > The second approach is less readable when looking at the tag, but it is > much more stable. Its as simple as that. One approach causes errors, the > other approach is more stable. Both point to Wikipedia article, just using > a slightly different URL internally. > > Automatically adding Wikidata tags is already being done by iD. I would > like to finish that process, so that the community can clean up all the > mistakes that are hiding in the OSM db. > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sun, Oct 1, 2017 at 8:15 PM, john whelanwrote: > Since an OSM object has lat and long value and it appears that wiki > whatever also has one the entries can be linked. > Not so. The data is very often different between wikipedia, wikidata, and OSM. Also, the same location could be a square, a famous sculpture within that square, and some commemorative plaque on it, and all could have some wikipedia/wikidata entry. Matching them up requires humans, and cannot reliably be done by an algorithm in a large number of cases. Lastly, if the coordinates are different, you may not copy it from OSM to Wikidata because of the difference in the license. > > "This gives you a very simple table with: lat/lon/page_title. > No parsing or anything else involved. > You then take data from OSM - lat/lon/wikipedia.tag > So you have two tables of same structure. Voila. You can compare > anything (title, coordinates), in any direction with some > approximation if needed etc. No OSM wikidata involved at all." > See above, this cannot be done with any reasonable reliability by automatic means. You will end up with an incredible amount of unreliable data. Feel free to discuss deleting of both Wikipedia and Wikidata tags, but I seriously doubt the community will go for it. > > I really don't understand why wikidata needs to be added. Note the word > need, I'm missing the requirement somehow that overides following normal > OSM practices. > Assuming my above arguments has convinced you -- that we must manually determine the match between an OSM feature and a Wikipedia article, lets discuss how best to link to Wikipedia. There are two options: link by article title, and link by Wikidata ID. The first one causes many errors - because titles get renamed, and old titles are reused for other meanings. The second approach is less readable when looking at the tag, but it is much more stable. Its as simple as that. One approach causes errors, the other approach is more stable. Both point to Wikipedia article, just using a slightly different URL internally. Automatically adding Wikidata tags is already being done by iD. I would like to finish that process, so that the community can clean up all the mistakes that are hiding in the OSM db. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Rather than fill OSM up with automated edits that have not even been discussed with the local community can we think more about functionality? Since an OSM object has lat and long value and it appears that wiki whatever also has one the entries can be linked. "This gives you a very simple table with: lat/lon/page_title. No parsing or anything else involved. You then take data from OSM - lat/lon/wikipedia.tag So you have two tables of same structure. Voila. You can compare anything (title, coordinates), in any direction with some approximation if needed etc. No OSM wikidata involved at all." I really don't understand why wikidata needs to be added. Note the word need, I'm missing the requirement somehow that overides following normal OSM practices. Cheerio John On 1 Oct 2017 7:19 pm,wrote: > Hi everybody. > > We already accepted wikipedia links keep in mind that wiki article isn't > the same abstraction as OSM object. > And the way we make a reference on wiki articles differs over time. > It was a link, it was an article name, it was a name with language prefix. > > Wikidata id is a way to make a refererence onto wikipedia article > which wiki community states to be the right and consistent one. > Why don't we just accept the recomended way of referencing wiki articles? > > That doesn't make wikipedia articles more or less verifable on the ground. > But that makes the link persistent agains the changes in wikipedia. > > 01.10.2017, 16:38, "Christoph Hormann" : > > On Sunday 01 October 2017, Marc Gemis wrote: > >> If I noticed an inception date on a information sign next to a > >> building, is this original research or a secondary source ? > > > > The date on the sign is verifiable as a signed date, not necessarily as > > a date connected to the building. Historic information like dates from > > before the timeframe witnessed by today's mappers is something that is > > problematic in OSM in general. > > > > As any historian will be able to tell you history as a science is not > > about analyzing events of the past in the same way you analyze present > > day observations in empiric science disciplines but about analyzing and > > interpreting sources - in this way it resembles the approach of > > Wikipedia more than that of OSM. > > > >> If a Wikidata contributor/Wikipedian researches websites like Orbis, > >> Onroerend erfgoed, etc and finds different inception dates and > >> records them all, is it bad that s/he uses secondary sources instead > >> of the information sign on the ground ? > > > > I don't want to judge the different approaches of Wikipedia and OSM - > > both have their pros and cons. As a contributor i am more comfortable > > with the OSM approach but i would never say that collecting information > > from secondary sources is inherently less valuable than collecting > > empiric data. > > > >> I wonder whether OSM is really always based on original research, or > >> whether we sometimes just want to believe this. > > > > Well - human perception is of course always connected to our past > > experience and beliefs, we cannot observe our environment in a truly > > neutral way. And of course with armchair mapping much of the data in > > OSM is produced by people who have never been near the place they map > > based on the belief that what the image layer they use depicts what is > > actually there (which is sometimes not the case - like with the iconic > > demolished buildings) and that what people think they see in the images > > actually resembles the impression they would have on the ground (which > > is also frequently not the case - prominent example would be here: > > http://www.openstreetmap.org/relation/7142214). But the key is every > > information in OSM has to be able to stand up to scrutiny by local > > mappers - a local mapper standing in front of the object needs to be > > able to tell if it is correct or not - even if different mappers could > > in borderline cases come to different assessments on the matter. > > > > -- > > Christoph Hormann > > http://www.imagico.de/ > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Hi everybody. We already accepted wikipedia links keep in mind that wiki article isn't the same abstraction as OSM object. And the way we make a reference on wiki articles differs over time. It was a link, it was an article name, it was a name with language prefix. Wikidata id is a way to make a refererence onto wikipedia article which wiki community states to be the right and consistent one. Why don't we just accept the recomended way of referencing wiki articles? That doesn't make wikipedia articles more or less verifable on the ground. But that makes the link persistent agains the changes in wikipedia. 01.10.2017, 16:38, "Christoph Hormann": > On Sunday 01 October 2017, Marc Gemis wrote: >> If I noticed an inception date on a information sign next to a >> building, is this original research or a secondary source ? > > The date on the sign is verifiable as a signed date, not necessarily as > a date connected to the building. Historic information like dates from > before the timeframe witnessed by today's mappers is something that is > problematic in OSM in general. > > As any historian will be able to tell you history as a science is not > about analyzing events of the past in the same way you analyze present > day observations in empiric science disciplines but about analyzing and > interpreting sources - in this way it resembles the approach of > Wikipedia more than that of OSM. > >> If a Wikidata contributor/Wikipedian researches websites like Orbis, >> Onroerend erfgoed, etc and finds different inception dates and >> records them all, is it bad that s/he uses secondary sources instead >> of the information sign on the ground ? > > I don't want to judge the different approaches of Wikipedia and OSM - > both have their pros and cons. As a contributor i am more comfortable > with the OSM approach but i would never say that collecting information > from secondary sources is inherently less valuable than collecting > empiric data. > >> I wonder whether OSM is really always based on original research, or >> whether we sometimes just want to believe this. > > Well - human perception is of course always connected to our past > experience and beliefs, we cannot observe our environment in a truly > neutral way. And of course with armchair mapping much of the data in > OSM is produced by people who have never been near the place they map > based on the belief that what the image layer they use depicts what is > actually there (which is sometimes not the case - like with the iconic > demolished buildings) and that what people think they see in the images > actually resembles the impression they would have on the ground (which > is also frequently not the case - prominent example would be here: > http://www.openstreetmap.org/relation/7142214). But the key is every > information in OSM has to be able to stand up to scrutiny by local > mappers - a local mapper standing in front of the object needs to be > able to tell if it is correct or not - even if different mappers could > in borderline cases come to different assessments on the matter. > > -- > Christoph Hormann > http://www.imagico.de/ > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 1 October 2017 at 18:29, Tomas Straupiswrote: >> Also, please elaborate which community has asked me to stay away??? > > Lithuania. Please can you point to the place where this was discussed and consensus reached, also to where that was communicated to the wider community? > We are in active action on not only fixing wikipedia > tags, but also adding missing tags to OSM, adding missing coordinates > to wikipedia, aligning coordinates between OSM and wikipedia etc. For > YEARS! Me too. That does not preclude the work Yuri is doing; the two are not mutually exclusive. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-10-01 21:45 GMT+02:00 Tomas Straupis: > > > >> When we create a POI detail page, we want to add a link (url without > >> redirects) to a wikipedia page. To do that it is straightforward to > >> use a value in wikipedia tag. > > Great, thanks. As you can see, nothing in what I do breaks that. > Instead, it > > actually helps your POI links to be more accurate. > > It does not help. We are not using wikidata in any way. We are > fixing wikipedia links, OSM objects, wikipedia articles manually using > automated checks described above to pinpoint the problems. > > *You* aren't using Wikidata. If you want a redirect to a wikipedia page to build an url, here it is https://www.wikidata.org/wiki/Special:GoToLinkedPage?site=enwiki=Q936 Instead of filling the database with hundreds of variations of wikipedia tag (wikipedia, wikipedia:, wikipedia=:* and so on) you need one to build the one you need. > -- > Tomas > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
>> It is mostly because you pushed the effort, not beaucse of >> "advantage of wikidata". The same fixing has already been done for >> YEARS before your effors based on wikipedia tags only. > > > Tomas, you claimed that "It adds NO value." This is demonstrably wrong. You > are right that the same fixing was done for years. But until wikidata tag, > there was no easy way to FIND them. There always was. You simply take wikipedia provided geo-tags dump like https://dumps.wikimedia.org/ltwiki/latest/ltwiki-latest-geo_tags.sql.gz This gives you a very simple table with: lat/lon/page_title. No parsing or anything else involved. You then take data from OSM - lat/lon/wikipedia.tag So you have two tables of same structure. Voila. You can compare anything (title, coordinates), in any direction with some approximation if needed etc. No OSM wikidata involved at all. If wikipedia page moves - title is gone from this dump and the new one appears on the same coordinates. You can map them very quickly. Theoretically you can update OSM data automatically, but usually if wikipedia title has changed, it means that something has changed in the object on the ground, so maybe something else has to be changed in OSM data as well (for example name). > About a year ago when I first started > this project, I created lists of thousands of such errors, that were very > rapidly fixed once they were identified. This was not possible before. Cool. I have nothing against that. I'm just saying the same could be done without wikidata tags. > My method is for finding broken wikipedia tags. What method are you talking > about? Can you describe what method you use to identify errors? See above. > Here is the DATA for my "theoretical ramblings". Can you show any data to > back your theoretical ramblings? See above. What are practical advantages of your method? Because theoretically you are taking a set A, creating a new set B from this A, and then you're trying to fix A according to B. This is logical nonsense :-) There is no point of putting this B into OSM. This is a temporary data which could be stored in your local "error checking" database. > Now there is a simpler http://tinyurl.com/ybv7q7n6 query - it used to have > about 1300, now down to ~750. And these are JUST the disambig errors. 550 objects globally... Well... :-) You should see from here, that the problem is finding people who want to FIX, not finding problems... > wiki. Lastly, what am I proposing to destroy?!? I am ADDING a tag and > ADDING a new search mechanism, because there is current no reliable > mechanism to fix these things. I have nothing against that. I'm arguing against idea that wikipedia tag is outdated or in any way worse. Yes, OSM would not be born without a geek idea, but it would not have reached what it is now if it would not be easy to understand for non geeks. Wikidata tag is totally non-understandable to non-geeks. > This is wonderful that you are fixing all these issues, could you tell me > how you find them? Also, funny enough, I used to live in Vilnius a long time > ago, near Gineitiškės. Should I be allowed to edit there? (I hope this > doesn't lead to another huge but unrelated discussion :) ) You know that is not the point. You could still live in Lithuania and you would still need to consult the local community before doing automated changes. >> When we create a POI detail page, we want to add a link (url without >> redirects) to a wikipedia page. To do that it is straightforward to >> use a value in wikipedia tag. > Great, thanks. As you can see, nothing in what I do breaks that. Instead, it > actually helps your POI links to be more accurate. It does not help. We are not using wikidata in any way. We are fixing wikipedia links, OSM objects, wikipedia articles manually using automated checks described above to pinpoint the problems. -- Tomas ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sun, Oct 1, 2017 at 1:29 PM, Tomas Straupiswrote: > 2017-10-01 20:04 GMT+03:00 Yuri Astrakhan: > >> 2. Its not a WORK to automatically update one osm tag according to > another > >> osm tag (anybody can do it online/locally/etc). It adds NO value. > > > > It adds HUGE value, as was repeatably shown. Thanks to Wikidata IDs, the > > community was able to see and fix tens of thousands of errors in > Wikipedia > > tags. <...> > > It is mostly because you pushed the effort, not beaucse of > "advantage of wikidata". The same fixing has already been done for > YEARS before your effors based on wikipedia tags only. Tomas, you claimed that "It adds NO value." This is demonstrably wrong. You are right that the same fixing was done for years. But until wikidata tag, there was no easy way to FIND them. About a year ago when I first started this project, I created lists of thousands of such errors, that were very rapidly fixed once they were identified. This was not possible before. > And fixing > wikipedia tags is in no way inferior to your method. Maybe even > better, because it involves less „geekiness“ - they are more > understandable to larger portion of OSM community. > > My method is for finding broken wikipedia tags. What method are you talking about? Can you describe what method you use to identify errors? >> 3. It is totally unacceptable to introduce idea that wikipedia tag could > >> be removed at some time, because some other new automatically filled > tag has > >> been introduced. > > > > First, it is always acceptable to introduce and discuss new ideas. Any > > ideas. Always. <...> > > Yes. But when you're told by numerous people numerous times that > current mechanism works, and there is nothing BETTER in your advice > (other than your theoretical rambilngs), you cannot advice to destroy > existing working mechanism. > Here is the DATA for my "theoretical ramblings". Can you show any data to back your theoretical ramblings? https://commons.wikimedia.org/w/index.php?title=Data: Sandbox/Yurik/OSM_objects_pointing_to_disambigs.tab=history Now there is a simpler http://tinyurl.com/ybv7q7n6 query - it used to have about 1300, now down to ~750. And these are JUST the disambig errors. There are many other types as I listed in the Wikipedia improvement project on osm wiki. Lastly, what am I proposing to destroy?!? I am ADDING a tag and ADDING a new search mechanism, because there is current no reliable mechanism to fix these things. > > We are discussing the way to improve them, > > because they are currently broken. Badly. > > And they are perfectly being fixed without involving wikidata tags > there, where people WANT to do that and do WORK to fix them. > > Do you have any data to back that up? When I first looked at them, Wikipedia links were often incorrect (see links above). Now they are fixed thanks to all the work done by the communities. Yes, all that manual work that people did. But in order to WORK, you need to FIND issues first. > > Also, please elaborate which community has asked me to stay away??? > > Lithuania. We are in active action on not only fixing wikipedia > tags, but also adding missing tags to OSM, adding missing coordinates > to wikipedia, aligning coordinates between OSM and wikipedia etc. For > YEARS! > This is wonderful that you are fixing all these issues, could you tell me how you find them? Also, funny enough, I used to live in Vilnius a long time ago, near Gineitiškės. Should I be allowed to edit there? (I hope this doesn't lead to another huge but unrelated discussion :) ) > > When we create a POI detail page, we want to add a link (url without > redirects) to a wikipedia page. To do that it is straightforward to > use a value in wikipedia tag. > Great, thanks. As you can see, nothing in what I do breaks that. Instead, it actually helps your POI links to be more accurate. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-10-01 20:04 GMT+03:00 Yuri Astrakhan: >> 2. Its not a WORK to automatically update one osm tag according to another >> osm tag (anybody can do it online/locally/etc). It adds NO value. > > It adds HUGE value, as was repeatably shown. Thanks to Wikidata IDs, the > community was able to see and fix tens of thousands of errors in Wikipedia > tags. <...> It is mostly because you pushed the effort, not beaucse of "advantage of wikidata". The same fixing has already been done for YEARS before your effors based on wikipedia tags only. And fixing wikipedia tags is in no way inferior to your method. Maybe even better, because it involves less „geekiness“ - they are more understandable to larger portion of OSM community. >> 3. It is totally unacceptable to introduce idea that wikipedia tag could >> be removed at some time, because some other new automatically filled tag has >> been introduced. > > First, it is always acceptable to introduce and discuss new ideas. Any > ideas. Always. <...> Yes. But when you're told by numerous people numerous times that current mechanism works, and there is nothing BETTER in your advice (other than your theoretical rambilngs), you cannot advice to destroy existing working mechanism. > We are discussing the way to improve them, > because they are currently broken. Badly. And they are perfectly being fixed without involving wikidata tags there, where people WANT to do that and do WORK to fix them. > Also, please elaborate which community has asked me to stay away??? Lithuania. We are in active action on not only fixing wikipedia tags, but also adding missing tags to OSM, adding missing coordinates to wikipedia, aligning coordinates between OSM and wikipedia etc. For YEARS! > 4) could you elaborate on who uses wikipedia > tags, and how they are being used? It would greatly help to understand > various use cases for such data. When we create a POI detail page, we want to add a link (url without redirects) to a wikipedia page. To do that it is straightforward to use a value in wikipedia tag. -- Tomas ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sun, Oct 1, 2017 at 11:12 AM, Tomas Straupiswrote: > I guess the point is that: > 1. Its ok to play with some pet-tag like wikidata > 100 % agree > 2. Its not a WORK to automatically update one osm tag according to another > osm tag (anybody can do it online/locally/etc). It adds NO value. > It adds HUGE value, as was repeatably shown. Thanks to Wikidata IDs, the community was able to see and fix tens of thousands of errors in Wikipedia tags. The Wikipedia link improvement project is based on it: https://wiki.openstreetmap.org/wiki/Wikipedia_Link_Improvement_Project Also, there is no point to add it online/locally because that wouldn't help community to find these errors. 3. It is totally unacceptable to introduce idea that wikipedia tag could be > removed at some time, because some other new automatically filled tag has > been introduced. > First, it is always acceptable to introduce and discuss new ideas. Any ideas. Always. We, as a community, don't have to accept them, but discussing innovations is always a good thing. That said, the removal of wikipedia tag is NOT being discussed here. We are discussing the way to improve them, because they are currently broken. Badly. > So if you like wikidata tag - go ahead and enjoy it, but do not tuch > wikipedia tag with autoscripts because people are actually using it. > Especially when you not only avoid discussing with local communities, but > ignore active requests from local communities to stay away. > > Tomas, 1) i don't have autoscripts to touch wikipedia tag, I use JOSM to generate wikidata tags because of the benefits it provides 2) i am providing a way for community to fix the broken wikipedia tags, 3) I have been very actively talking to many communities (in, ru, de, fr, ...). Also, please elaborate which community has asked me to stay away??? A very broad statement, considering that every single community had many members supporting this effort. And 4) could you elaborate on who uses wikipedia tags, and how they are being used? It would greatly help to understand various use cases for such data. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
I guess the point is that: 1. Its ok to play with some pet-tag like wikidata 2. Its not a WORK to automatically update one osm tag according to another osm tag (anybody can do it online/locally/etc). It adds NO value. 3. It is totally unacceptable to introduce idea that wikipedia tag could be removed at some time, because some other new automatically filled tag has been introduced. So if you like wikidata tag - go ahead and enjoy it, but do not tuch wikipedia tag with autoscripts because people are actually using it. Especially when you not only avoid discussing with local communities, but ignore active requests from local communities to stay away. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 1 October 2017 at 14:03, Andy Townsendwrote: > On 01/10/2017 13:05, Andy Mabbett wrote: >> And now you're making things up. >> > just two posts earlier in this thread you said > >> I don't think this kind of sarcastic hyperbole helps to move things >> forward, nor to engender an atmosphere of collegial collaboration. >> Please resist the temptation to use it. > > I would respectfully suggest that you follow your own advice. I do; my comment was not "sarcastic hyperbole", but a factual observation. > There's a valid discussion to be had about "how OSM does things vs how > wikipedia/wikidata does things" Has anyone said there is not? -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 01/10/2017 13:05, Andy Mabbett wrote: And now you're making things up. just two posts earlier in this thread you said > I don't think this kind of sarcastic hyperbole helps to move things > forward, nor to engender an atmosphere of collegial collaboration. > Please resist the temptation to use it. I would respectfully suggest that you follow your own advice. There's a valid discussion to be had about "how OSM does things vs how wikipedia/wikidata does things". Back in 2016 in another context I mentioned "Common End" in Derbyshire on this list https://lists.openstreetmap.org/pipermail/talk/2016-November/077139.html . It's in wikipedia as "a place noted on a map" (which is correct - OS maps include it). It doesn't in any verifiable sense "exist" though. Wikidata has it https://www.wikidata.org/wiki/Q5153341 as both a "hamlet" and a "fictional location". Whether that's "correct" or not is a decision for wikidata - I've no idea what their definition of "hamlet" is and whether it includes a locality that probably used to exist in some sense but all on-the-ground trace of the name has disappeared, but it's entirely reasonable to discuss the areas in which different contribution customs will result in different data, and how we handle links in those cases. Best Regards, Andy ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 1 October 2017 at 12:13, Christoph Hormannwrote: > the consensus in the OSM community to link to Wikipedia articles, and > other useful third-party sources? > I find it fascinating how both you and Yuri seem to be eager to always > deflect the discussion from the main subject, namely the automated > editing/addition of wikidata IDs Your 337-word post to which I replied, mentioned Wikidata only once, tangentially, mentioned Wikipedia nine times, and was mostly concerned with your thesis on your perception of "the fundamental differences between OSM and Wikipedia." > and misinterpreting constructive > critique of that as an attempt to tell local mappers what tags they may > and may not add to the things they map. And now you're making things up. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sunday 01 October 2017, Marc Gemis wrote: > > If I noticed an inception date on a information sign next to a > building, is this original research or a secondary source ? The date on the sign is verifiable as a signed date, not necessarily as a date connected to the building. Historic information like dates from before the timeframe witnessed by today's mappers is something that is problematic in OSM in general. As any historian will be able to tell you history as a science is not about analyzing events of the past in the same way you analyze present day observations in empiric science disciplines but about analyzing and interpreting sources - in this way it resembles the approach of Wikipedia more than that of OSM. > If a Wikidata contributor/Wikipedian researches websites like Orbis, > Onroerend erfgoed, etc and finds different inception dates and > records them all, is it bad that s/he uses secondary sources instead > of the information sign on the ground ? I don't want to judge the different approaches of Wikipedia and OSM - both have their pros and cons. As a contributor i am more comfortable with the OSM approach but i would never say that collecting information from secondary sources is inherently less valuable than collecting empiric data. > I wonder whether OSM is really always based on original research, or > whether we sometimes just want to believe this. Well - human perception is of course always connected to our past experience and beliefs, we cannot observe our environment in a truly neutral way. And of course with armchair mapping much of the data in OSM is produced by people who have never been near the place they map based on the belief that what the image layer they use depicts what is actually there (which is sometimes not the case - like with the iconic demolished buildings) and that what people think they see in the images actually resembles the impression they would have on the ground (which is also frequently not the case - prominent example would be here: http://www.openstreetmap.org/relation/7142214). But the key is every information in OSM has to be able to stand up to scrutiny by local mappers - a local mapper standing in front of the object needs to be able to tell if it is correct or not - even if different mappers could in borderline cases come to different assessments on the matter. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sunday 01 October 2017, Andy Mabbett wrote: > > Or perhaps it is you who is "deep denial" about the consensus in the > OSM community to link to Wikipedia articles, and other useful > third-party sources? May i suggest you to read my previous messages on this thread to find the answer to that question? I find it fascinating how both you and Yuri seem to be eager to always deflect the discussion from the main subject, namely the automated editing/addition of wikidata IDs and misinterpreting constructive critique of that as an attempt to tell local mappers what tags they may and may not add to the things they map. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 1 October 2017 at 10:06, Christoph Hormannwrote: > On Sunday 01 October 2017, Yuri Astrakhan wrote: > Wow, i had to read this twice to really believe what i was reading here. I don't think this kind of sarcastic hyperbole helps to move things forward, nor to engender an atmosphere of collegial collaboration. Please resist the temptation to use it. > Seems you are still in deep denial about the fundamental differences > between OSM and Wikipedia. Or perhaps it is you who is "deep denial" about the consensus in the OSM community to link to Wikipedia articles, and other useful third-party sources? -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> Wikipedia is based on secondary sources, it rejects original research. > OSM is fundamentally different in that because it is based on > verification by original research. I'm trying to understand this. If I noticed an inception date on a information sign next to a building, is this original research or a secondary source ? If a Wikidata contributor/Wikipedian researches websites like Orbis, Onroerend erfgoed, etc and finds different inception dates and records them all, is it bad that s/he uses secondary sources instead of the information sign on the ground ? I wonder whether OSM is really always based on original research, or whether we sometimes just want to believe this. regards m ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Christoph, I am not talking about OSM or Wikidata or Wikipedia quality or approaches. Please don't read more into it than what I am trying to state. If we say that we want OSM objects to link to Wikipedia (and we clearly do, judging by the number of wikipedia tags people have created), we need a good way to do it. Linking to Wikipedia with the page titles is bad. It is not stable. Wikidata tags fixes that. No other claim is being made here. On Sun, Oct 1, 2017 at 5:06 AM, Christoph Hormannwrote: > On Sunday 01 October 2017, Yuri Astrakhan wrote: > > > > Wikipedia created a stable ID system for these pages. Its called > > Wikidata. Please view Wikidata as first and foremost a linking system > > to Wikipedia articles. [...] > > > > Andy, you keep saying Wikidata is not verifiable data - but that's > > because you keep insisting on separating it from Wikipedia. > > Wow, i had to read this twice to really believe what i was reading here. > Seems you are still in deep denial about the fundamental differences > between OSM and Wikipedia. > > Wikipedia is based on secondary sources, it rejects original research. > Therefore you can find a lot of nonsense on Wikipedia - all kind of > urban legends and things like that, especially about remote areas, as > long as everyone believes them and no one bothers to proof them wrong > and rebut them outside of Wikipedia. So in a way Wikipedia documents > societies current beliefs about the world, not the world itself. This > does not necessarily have to go as far as an article about something > fictitious claiming to be about a real world thing, often its smaller > stuff like X being an object of type Y. The iconic 'citation needed' > of Wikipedia is not about the information being in need of actual > verification as a fact, it is about this information being verified to > be something well integrated into societies' belief system. > > OSM is fundamentally different in that because it is based on > verification by original research. This does not mean everything in > OSM holds up to this standard but we aim for this and value information > that is practically verifiable by local mappers and tagging concepts > that are targeted at verifiable mapping more than other information > that people always will keep adding to OSM to some extent despite it > being non-verifiable. > > It also means information in OSM is inherently more variable because > what people observe on the ground varies - both because what people see > depends on their experience and background and because appearance of > reality, especially of natural features, varies over time. OSM with > its original research research focus lacks the unifying and consistency > preserving effect of the filter through secondary sources you have in > Wikipedia. > > What you do when you mechanically 'fix errors' and correct discrepancies > between tags in OSM that contradict the Wikipedia/Wikidata information > is you impose the value system of Wikipedia onto OSM. > > -- > Christoph Hormann > http://www.imagico.de/ > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Sunday 01 October 2017, Yuri Astrakhan wrote: > > Wikipedia created a stable ID system for these pages. Its called > Wikidata. Please view Wikidata as first and foremost a linking system > to Wikipedia articles. [...] > > Andy, you keep saying Wikidata is not verifiable data - but that's > because you keep insisting on separating it from Wikipedia. Wow, i had to read this twice to really believe what i was reading here. Seems you are still in deep denial about the fundamental differences between OSM and Wikipedia. Wikipedia is based on secondary sources, it rejects original research. Therefore you can find a lot of nonsense on Wikipedia - all kind of urban legends and things like that, especially about remote areas, as long as everyone believes them and no one bothers to proof them wrong and rebut them outside of Wikipedia. So in a way Wikipedia documents societies current beliefs about the world, not the world itself. This does not necessarily have to go as far as an article about something fictitious claiming to be about a real world thing, often its smaller stuff like X being an object of type Y. The iconic 'citation needed' of Wikipedia is not about the information being in need of actual verification as a fact, it is about this information being verified to be something well integrated into societies' belief system. OSM is fundamentally different in that because it is based on verification by original research. This does not mean everything in OSM holds up to this standard but we aim for this and value information that is practically verifiable by local mappers and tagging concepts that are targeted at verifiable mapping more than other information that people always will keep adding to OSM to some extent despite it being non-verifiable. It also means information in OSM is inherently more variable because what people observe on the ground varies - both because what people see depends on their experience and background and because appearance of reality, especially of natural features, varies over time. OSM with its original research research focus lacks the unifying and consistency preserving effect of the filter through secondary sources you have in Wikipedia. What you do when you mechanically 'fix errors' and correct discrepancies between tags in OSM that contradict the Wikipedia/Wikidata information is you impose the value system of Wikipedia onto OSM. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Verifiability is critical to OSM success, but it does not mean it must only be verifiable by visiting the physical location. Tags like "wikipedia", "wikidata", "url", "website" and some IDs cannot be verified that way. You must visit some external website to validate. Stopping by Yellowstone National Park or a statue in the middle of a city may tell you its national registration number, but most likely you will have to visit some government website. Seeing some complex URL tells you nothing about its correctness unless you visit that web site. Yet, we are not talking about the last two examples. Node 153699914 has wikipedia="Eureka, Wisconsin". It looks fine to a casual examiner, but in reality is a garbage link to a disambiguation place - a list of 3 different places, which you wouldn't know unless you visit the external site - Wikipedia. I have uncovered many thousands of such cases, and many of them have already been fixed thanks to a stronger IDing system. Yet, every day there is more of them - because Wikipedia keeps renaming things, and several people refuse to allow Wikidata IDs. Wikipedia created a stable ID system for these pages. Its called Wikidata. Please view Wikidata as first and foremost a linking system to Wikipedia articles. It is NOT perfect. It has many issues. But it is simply much better than linking to Wikipedia articles by their names because they don't break as often. Andy, you keep saying Wikidata is not verifiable data - but that's because you keep insisting on separating it from Wikipedia. We can already make it so that when you click on Wikidata link, you are taken directly to Wikipedia. The statements on Wikidata entries are a major bonus for automated verification and other things, but it should be viewed in addition to the redirecting capability, not as a replacement to Wikipedia pages. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 28/09/2017 07:15, Stefano wrote: We used this library to process the dump and then we add the results in pgsql https://www.entropywins.wtf/blog/2015/11/08/wikidata-wikibase-json-dump-reader/ https://github.com/osmItalia/wikidata-geo-match What would the requirement of a wikidata2pgsql be? Thanks for that. To answer the question, It'd create a database in a format that's designed to be queried that contains "just enough" information to support whatever job it's needed for (and it'd be great if it also supported dynamic column creation using a mechanism similar to osm2pgsql's ".style" file). Of the other "missing bits", I used "osmosis" as an example of "cutting a database extract down to size" (other options are available). In the OSM world that initial slice is often geographical, but osmosis can also deal with data without explicit co-ordinates (ways and relations) based on the geographical location of constituent nodes. The same would be true (for me) of wikidata - I'd be only interested in actual physical locations and the things that they link to (which may not have physical locations and may just be concepts). Finally "switch2osm" is a regularly-updated set of instructions that you can follow from start to finish without needed external knowledge about how to solve a problem. For example, https://switch2osm.org/manually-building-a-tile-server-16-04-2-lts/ has undergone numerous updates in the last year to deal with stylesheet changes, which depend on a bleeding edge version of carto, which depends on node.js. At each stage in the process the idea was that you'd always be able to get a working result, even if at one point that meant the instructions explained how get a version of the stylesheet from a few months ago because a newer version wouldn't work in combination with everything else there. "wikidata-wikibase-json-dump-reader" looks interesting - it looks (to continue the analogy) to be somewhat equivalent to the Crosby PBF library. "wikidata-geo-match" also looks interesting because https://github.com/osmItalia/wikidata-geo-match/blob/master/scripts/2_process.sh and the readme explain how to do the initial geographical selection. It's not quite all there though (https://github.com/osmItalia/wikidata-geo-match/search?utf8=%E2%9C%93=0_create_wikidata_table.sql= suggests either the README is out of date or some bits are missing). It'd certainly be a useful start for someone who cared about wikidata to develop something mirroring the equivalent tools that OSM already has, and I'm sure it does exactly what you need it to do, but it's not a generic "let's create a database and allow you to do something with it". Best Regards, Andy ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 28 Sep 2017 2:57 am, "Andy Townsend"wrote: It depends - if you want to do a "quick search for something" then an equivalent to overpass turbo might be an option, but in the real world what you'd _actually_ want to do is a local database query. Unfortunately that side of things seems to be completely missing (or at least very well-hidden) - wikidata seems to be quite immature in that respect. Where's the "switch2osm" for wikidata? Where's the "osm2pgsql" or "osmosis"? Sure I can download 20Gb of gzipped JSON from https://dumps.wikimedia.org/wikidatawiki/entities/20170925/ and try and write some sort of parser based on https://www.mediawiki.org/ wiki/Wikibase/DataModel/JSON , but this seems very much like going back to banging the rocks together (and no, a third-party query interface that depends on an external network connection such as https://query.wikidata.org/ or anything else isn't a better option). We used this library to process the dump and then we add the results in pgsql https://www.entropywins.wtf/blog/2015/11/08/wikidata-wikibase-json-dump-reader/ https://github.com/osmItalia/wikidata-geo-match What would the requirement of a wikidata2pgsql be? Regards, Andy Ciao, Stefano ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 26/09/2017 18:08, Yuri Astrakhan wrote: When data consumers want to get a link to corresponding wikipedia article, doing that with wikipedia[:xx] tags is straightforward. Doing the same with wikidata requires additional pointless and time consuming abrakadabra. no, you clearly haven't worked with any data consumers recently. Data consumers want Wikidata, much more than wikipedia tags - please talk to them. That would be me in a former job, I think. One of the things that I used to spend a lot of time doing was finding ways to encode data so that knowledge could be shared by e.g. field engineers, and then analysing those results so that you can find out what was related to what, what caused what, and how much store you can set by a particular result or prediction. There are a couple of points worth sharing from that experience: 1) The first point to make about human-contributed data is that it's variable. Some people will say something is probably an X, some people probably a Y. The reality is that they're actually both right some of the time. You might think (in the context of e.g. shop brands) "hang on - surely a shop can be only one brand? It must be _either_ X or Y!" but you'd be wrong. There are _always_ exceptions, and there will always be "errors" - you just don't know which way is right and which wrong. 2) The second point that's relevant here is that codes such as CODE1, CODE2 etc. are to be avoided at all costs since they don't enable any natural visualisation of what's been captured. You have already said "but surely every system that displays data can look up the description" but anyone familar with the real world knows that that simply won't happen. This means that there's no way for an ordinary mapper to verify whether the magic code on an OSM item is correct or not. Verifiability is one of the key concepts of OSM (see https://wiki.openstreetmap.org/wiki/Verifiability et al) and anything that moves away from it means that data isn't going to be maintained, because people simply won't understand what it means. I suspect that a key part of the success of OSM was the reliance on natural language-based keys and values, and a loose tagging scheme that allowed easy expansion. 3) The third point is that a database that has been "cleaned" so that there are no "errors" in it is worth far less than one that hasn't, when you're trying to understand the complex relationships between objects. This goes against most normal data processing instincts because obviously normally you'd try and ensure that data has full referential integrity - but where there are edge cases (and as per (1) above there are always edge cases) different consumers will very likely want to treat those edge cases differently, which they can't do if someone has "helpfully" merged all the edge cases into more popular categories. To be blunt, if I was trying to process OSM data and had a need to get into the wikidata/wikipedia world based on it (for example because I wanted the municipal coat of arms - something not in OSM) I'd take a wikipedia link over a wikidata one every time because all mappers will have been able to see the text of the wikipedia link rather than just something like Q123456. You've made the point that things change in wikipedia regularly (articles get renamed etc.), but it's important to remember that things change in the real world all the time as well - and a link that's suddenly pointing at something different in wikipedia is immediately apparent, in the same way that if Q123456 was no longer relevant (because the real world thing has changed) it wouldn't be. All that said, I don't see wikidata as a key component (or even a very useful component) of OSM - but we all map things that are of interest to us - some people map in great detail the style of British telephone boxes or the "Royal Cipher" on postboxes which I see absolutely no point in, but if it's verifiable, why not - I'm sure I'm mapping stuff that is irrelevent to them. A problem with wikidata (as noted above) is that I'm not sure that it _is_ verifiable data - I suspect it'll get stale after adding and never be maintained, simply because people will never notice that it's wrong. (and on an unrelated comment in the same message) Sure, it can be via dump parsing, but it is a much more complicated than querying. Would you rather use Overpass turbo to do a quick search for some weird thing that you noticed, or download and parse the dump? Most people would rather do the former. It depends - if you want to do a "quick search for something" then an equivalent to overpass turbo might be an option, but in the real world what you'd _actually_ want to do is a local database query. Unfortunately that side of things seems to be completely missing (or at least very well-hidden) - wikidata seems to be quite immature in that respect. Where's
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
I have been fixing nodes that have wikipedia but no wikidata tags [1], and even the first two randomly picked nodes had identical problem - article was renamed (twice!) without leaving redirects - node 1136510320 Try it yourself - run the query and see what the it points to. [1] https://wiki.openstreetmap.org/wiki/Wikipedia_Link_Improvement_Project#Missing_Wikidata_tags Imre, I think at this point it might be better to have both, just as a safety check. But I can already see that they get misaligned - articles keep getting renamed, so we will be stuck mindlessly updating wikipedia tag. Feels a bit like a busywork for the sake of work, but might be needed for a bit. On Wed, Sep 27, 2017 at 6:00 PM, Imre Samuwrote: > > I hope everyone realizes that there are Wikidata items for which there > > is no Wikipedia article. > > So you cannot always find it via Wikipedia tags. > > And at least JOSM shows a human readable name of a Wikidata item > > besides the Q-number. I think iD does this as well. > > m. (who manually adds Wikidata references for Flemish churches after > creating the Wikidata items). > > imho: > probably you have a local and domain knowledge on the topic of "Flemish > churches" > but for me: wikidata without wikipedia page - is extremely suspicious > > because: > > #1. Sometimes the " nearby" search for geolocated articles/wikidataids is > not enough > for example: > * at least ~28000 churches exist in the wikidata without coordinates: > http://tinyurl.com/y8nyk9zw > > And probably we will also find wikidata cities without coordinates. > > #2. And we should aware of the current "Parallel geo worlds" problem in > the wikidata[1] > for example: > Arad ( major City in Romania ) has 3 wikidata, and we should prefer id > with wikipedia pages. > * https://www.wikidata.org/wiki/Q173591 ( with wikipedia pages, linked to > OSM ) > * https://www.wikidata.org/wiki/Q31886684 ( created by Cebuano import > [1] ~1 month ago) * https://www.wikidata.org/wiki/Q16898082 > > [1] wikidata cebuano import problem: > * https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/ > 2017/08#Dealing_with_our_second_planet * https://www.wikidata.org/wiki/ > Wikidata:Project_chat/Archive/2017/08#Nonsense_imported_from_Geonames > > > Imre > > > > > > > 2017-09-27 5:03 GMT+02:00 Marc Gemis : > >> On Wed, Sep 27, 2017 at 1:17 AM, Andy Townsend wrote: >> > That's simply rubbish. Tags on an OSM object describe it in the real >> world. >> > They should be verifiable. Whether an OSM object has a wikidata tag on >> it >> > is essentially irrelevant as far as OSM is concerned - it's just a >> primary >> > key into an external database. External data consumers might find the >> data >> > in that database useful, but they can also get to it via wikipedia tags >> > (which, being human-readable, are more likely to be maintained), so it's >> > really not a big deal. >> >> >> I hope everyone realizes that there are Wikidata items for which there >> is no Wikipedia article. So you cannot always find it via Wikipedia >> tags. >> >> And at least JOSM shows a human readable name of a Wikidata item >> besides the Q-number. I think iD does this as well. >> >> m. (who manually adds Wikidata references for Flemish churches after >> creating the Wikidata items). >> >> ___ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk >> > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> I hope everyone realizes that there are Wikidata items for which there > is no Wikipedia article. > So you cannot always find it via Wikipedia tags. > And at least JOSM shows a human readable name of a Wikidata item > besides the Q-number. I think iD does this as well. > m. (who manually adds Wikidata references for Flemish churches after creating the Wikidata items). imho: probably you have a local and domain knowledge on the topic of "Flemish churches" but for me: wikidata without wikipedia page - is extremely suspicious because: #1. Sometimes the " nearby" search for geolocated articles/wikidataids is not enough for example: * at least ~28000 churches exist in the wikidata without coordinates: http://tinyurl.com/y8nyk9zw And probably we will also find wikidata cities without coordinates. #2. And we should aware of the current "Parallel geo worlds" problem in the wikidata[1] for example: Arad ( major City in Romania ) has 3 wikidata, and we should prefer id with wikipedia pages. * https://www.wikidata.org/wiki/Q173591 ( with wikipedia pages, linked to OSM ) * https://www.wikidata.org/wiki/Q31886684 ( created by Cebuano import [1] ~1 month ago) * https://www.wikidata.org/wiki/Q16898082 [1] wikidata cebuano import problem: * https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/ 2017/08#Dealing_with_our_second_planet * https://www.wikidata.org/wiki/ Wikidata:Project_chat/Archive/2017/08#Nonsense_imported_from_Geonames Imre 2017-09-27 5:03 GMT+02:00 Marc Gemis: > On Wed, Sep 27, 2017 at 1:17 AM, Andy Townsend wrote: > > That's simply rubbish. Tags on an OSM object describe it in the real > world. > > They should be verifiable. Whether an OSM object has a wikidata tag on > it > > is essentially irrelevant as far as OSM is concerned - it's just a > primary > > key into an external database. External data consumers might find the > data > > in that database useful, but they can also get to it via wikipedia tags > > (which, being human-readable, are more likely to be maintained), so it's > > really not a big deal. > > > I hope everyone realizes that there are Wikidata items for which there > is no Wikipedia article. So you cannot always find it via Wikipedia > tags. > > And at least JOSM shows a human readable name of a Wikidata item > besides the Q-number. I think iD does this as well. > > m. (who manually adds Wikidata references for Flemish churches after > creating the Wikidata items). > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
So you do not agree with the Automated Edits code of conduct ? If an automated edit takes place in a country, why do you expect that that community follows the talk mailing list or even speak English ? People has the right to know that some stranger starts making changes in their area without being forced to read a mailing list (which is an outdated medium for the younger) or understand English. An import has to be discussed with the local community, so why would an automated edit be different ? regards m. On Wed, Sep 27, 2017 at 5:38 PM, Martin Koppenhoeferwrote: > > > 2017-09-27 16:56 GMT+02:00 Andy Mabbett : >> >> On 26 September 2017 at 21:39, Martin Koppenhoefer >> wrote: >> >> >> This might also mean that >> >> you have to discuss it via Telegram, Facebook, email, IRC, etc. >> >> depending on where that local community is. >> >> >> >> The talk mailing list is not sufficient. >> >> > I think this is problematic. If the local community uses a paid service >> > for >> > communication you'd have to pay in order to speak to your fellow >> > mappers? >> >> The mapping community in the West Midlands of England communicates >> most often face-to-face, meeting in a a pub. >> >> Perhaps we could mandate that anyone wanting to make an automated edit >> in the region has to buy the beer? > > > > you don't only pay with money, e.g. in Facebook you only pay money if you > are a client of theirs (buying visibility for your advertizing), the > ordinary users (merchandise) pay with data and their privacy. > > Cheers, > Martin > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-09-27 16:56 GMT+02:00 Andy Mabbett: > On 26 September 2017 at 21:39, Martin Koppenhoefer > wrote: > > >> This might also mean that > >> you have to discuss it via Telegram, Facebook, email, IRC, etc. > >> depending on where that local community is. > >> > >> The talk mailing list is not sufficient. > > > I think this is problematic. If the local community uses a paid service > for > > communication you'd have to pay in order to speak to your fellow mappers? > > The mapping community in the West Midlands of England communicates > most often face-to-face, meeting in a a pub. > > Perhaps we could mandate that anyone wanting to make an automated edit > in the region has to buy the beer? > you don't only pay with money, e.g. in Facebook you only pay money if you are a client of theirs (buying visibility for your advertizing), the ordinary users (merchandise) pay with data and their privacy. Cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 26 September 2017 at 21:39, Martin Koppenhoeferwrote: >> This might also mean that >> you have to discuss it via Telegram, Facebook, email, IRC, etc. >> depending on where that local community is. >> >> The talk mailing list is not sufficient. > I think this is problematic. If the local community uses a paid service for > communication you'd have to pay in order to speak to your fellow mappers? The mapping community in the West Midlands of England communicates most often face-to-face, meeting in a a pub. Perhaps we could mandate that anyone wanting to make an automated edit in the region has to buy the beer? -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wed, Sep 27, 2017 at 1:17 AM, Andy Townsendwrote: > That's simply rubbish. Tags on an OSM object describe it in the real world. > They should be verifiable. Whether an OSM object has a wikidata tag on it > is essentially irrelevant as far as OSM is concerned - it's just a primary > key into an external database. External data consumers might find the data > in that database useful, but they can also get to it via wikipedia tags > (which, being human-readable, are more likely to be maintained), so it's > really not a big deal. I hope everyone realizes that there are Wikidata items for which there is no Wikipedia article. So you cannot always find it via Wikipedia tags. And at least JOSM shows a human readable name of a Wikidata item besides the Q-number. I think iD does this as well. m. (who manually adds Wikidata references for Flemish churches after creating the Wikidata items). ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 26/09/2017 03:40, Yuri Astrakhan wrote: ... I have been blocked by Andy Townsend with the following message. Let's begin at the beginning - this was a "0-hour block" - you weren't prevented from using the API for _any_ period of time, merely forced to read this message first. This was a last resort - many other attempts at communication have been made over at least the last 10 months (since November 2016 - https://www.openstreetmap.org/changeset/43749373 ). The issues that I raised back then are still true today - see https://lists.openstreetmap.org/pipermail/talk/2017-September/078780.html for more details. It makes no sense to mechanically copy a wikidata value to OSM when the wikidata object expresses only part of the sense of the wikipedia page. Simple example in case things are still not clear: 1) Imagine there are two objects in OSM - a village and an admin area containing that village. 2) Wikipedia only has one page for a "village and an admin area" 3) The wikidata page (probably created by a bot) is only for the village 4) Linking the OSM admin area to the wikidata page for the village is an error. This is the sort of thing that you've been doing again and again for the last 10 months. A few interesting semi-relevant statistics so far: the number of discovered links to disambig pages is now back to over 800, even without 100k+ untaged ways. And there are almost 38,000 osm objects where wikipedia tag does not correspond with wikidata tag. The number is very high, but fixing them should be semi-automated, as most of them are redirects. TBD. There are a lots of possibilities here. Maybe the OSM object shouldn't have a wikipedia entry at all. Maybe it's significantly changed since the link was added, and should be changed. It needs someone with real-world knowledge of the OSM object to update the links - anything else is just guessing, and has no place in OSM. If by "semi-automated" you mean a human-centric approach like Kort, MapRoulette, StreetComplete et al then fine - but that's not been your approach so far. Here's Andy's message, with my inlined replies. I think that almost all of the raised points have been raised and answered in our previous discussion, but I feel it is my responsibility to present them again. You're conducting an import of known bad data (your own changeset comments say "Further cleanup will be done using..."). Per previous description, the existing data is already bad, and I am simply making it possible to identify it, after discussing it on this thread. No, that is untrue. See e.g. https://www.openstreetmap.org/changeset/52002597 . You are wilfully ignoring the feedback that you're receiving now and have received in the past. A lot of issues have been raised about the quality of your edits - see http://resultmaps.neis-one.org/osm-discussion-comments?uid=339581 . In many cases you seem to agree that you're adding rubbish, and yet you continue. You seem to be suggesting (in https://lists.openstreetmap.org/pipermail/talk/2017-September/078767.html ) that "the community" clean up your mess. This is not the way that OpenStreetMap works - if an individual is adding data to it (especially large quantities of data) then it is their responsibility to ensure that the data that they are adding is valid, or at least as valid as the data that is already there. Again, no, I am identifying rubbish, not introducing it, and I am very actively replying to every comment I receive. You are not actually _resolving_ any of the problems that people are finding with the edits that you are making. See for example https://www.openstreetmap.org/changeset/52341792 . In that example someone says that you added a wikidata tag in error. You agree that you added it in error (and in fact a whole category of the tags that you've added is in error - I've commented on a couple more within the last hour). You have not done anything to resolve this error that you have introduced into OSM . Going further back, in your replies to changeset comments you've said things like "I have already stopped changing any objects except the admin levels regions 1-6" https://openstreetmap.org/changeset/4377 but have carried on regardless. Mappers have repeatedly asked you to use geographically smaller changesets https://openstreetmap.org/changeset/44078387**https://openstreetmap.org/changeset/44090685 https://openstreetmap.org/changeset/44203236 and yet you continue regardless. Either you're incompetent in the changes you're making or you're lying to us; in neither case should you be continuing to edit as you have been doing. ... The way to solve the quality of this data is to analyze it with the OSM+Wikidata tool I have built, ... or with something else that doesn't require OSM to be mechanically edited by you first. As has already been said
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-09-26 5:14 GMT+02:00 Marc Gemis: > This might also mean that > you have to discuss it via Telegram, Facebook, email, IRC, etc. > depending on where that local community is. > > The talk mailing list is not sufficient. I think this is problematic. If the local community uses a paid service for communication you'd have to pay in order to speak to your fellow mappers? Cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-09-26 19:43 GMT+02:00 Yves: > I think that the underlying issue in wikidata tags is that they are > external IDs. Not human readable, they cannot be entered 'by hand' nor > verified on the ground. > Yes, it's an external ID, but it acts as intermediary with other databases, because Wikidata acts as a central repository for IDs (see https://cacm.acm.org/magazines/2014/10/178785-wikidata/fulltext). One could perform a reconciliation of its database against Wikidata instead of against OSM, and get a match 'for free'. We can't add OSM ids in WD because they aren't stable, so Wikidata IDs could be the stable IDs for certain classes of objects (those "worth" of some description in another project?). "Entered by hand" they can't be in the same way as Wikipedia entries can't (you add a wikipedia tag hoping someone will fill the "red link" on that wikipedia page?) > Once you accept them in OSM, you can't really complain about bots. > > Yves (who still think such UIDs are only needed for the lack of good query > tools). > > > Stefano ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Yves, yes, they are external IDs. But so are wikipedia titles. Visually inspecting Wikipedia tile does not provide you with any way to verify its correctness - you have to look in the external data source (WP). As for entering by hand - just like you shouldn't enter Wikipedia articles by hand - you should copy/paste it from the article, or use the autocomplete field in iD. So in reality, these two things are nearly the same. On the other hand, modern rely on the internet connection, which means that an ID can be shown as text in the user's language, together with other metadata from Wikidata. The concept of "internal" vs "external" is not as relevant now as it was in the past... (there is only one data - the internet :)) On Tue, Sep 26, 2017 at 1:43 PM, Yveswrote: > I think that the underlying issue in wikidata tags is that they are > external IDs. Not human readable, they cannot be entered 'by hand' nor > verified on the ground. > Once you accept them in OSM, you can't really complain about bots. > > Yves (who still think such UIDs are only needed for the lack of good query > tools). > > > > Le 26 septembre 2017 19:08:33 GMT+02:00, Yuri Astrakhan < > yuriastrak...@gmail.com> a écrit : >> >> > p.s. OSM is a community project, not a programmers project, it's about >>> > people, not software :-) >>> >>> It's both. OSM is first and foremost is a community, but the result of >> our effort is a machine-readable database. We are not creating an >> encyclopedia that will be casually flipped through by humans. We produce >> data that gets interpreted by software, so that it can render maps and be >> searchable. For example, if every person uses their own tag names and ways >> to record things, the data will have nearly zero value. We must agree on >> conventions so that software can understand our results - which is exactly >> what we have been doing on wiki and in email channels. Any tag and value >> that cannot be recognized and processed by software is effectively ignored. >> >> >>> Totally agree. If some script can automatically add new tag >>> (wikidata) without any actual WORK needed, then it is pointless, >>> anybody can run an auto-update script. >> >> When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia >>> data, they add wikipedia link, not wikidata "stuff". >>> >> >> While sand castles may look nice, they don't last very long. When >> ordinary people add just the Wikipedia article, that link quickly gets >> stale and become irrelevant and often incorrect. The wikipedia article >> titles are not stable. They get renamed all the time - there are tens of >> thousands of them in OSM already that I found. Often, community renames wp >> articles because there are more than one meaning, so they create a new >> article with the same name in its place - a disambig page. There is no >> easy way to analyse wikipedia links for content - you cannot easily >> determine if the wikipedia article is about a person, a country, or a >> house, which makes it impossible to check for correctness. >> >> When I spend half an hour of my time researching which WP article is best >> for an object, I do not want that effort to be wasted just because someone >> else puts a disambig page in its place, and I have to redo all my work. >> >> When data consumers want to get a link to corresponding wikipedia >>> article, doing that with wikipedia[:xx] tags is straightforward. Doing >>> the same with wikidata requires additional pointless and time >>> consuming abrakadabra. >>> >> >> no, you clearly haven't worked with any data consumers recently. Data >> consumers want Wikidata, much more than wikipedia tags - please talk to >> them. Wikidata gives you the list of wikipedia articles in all available >> languages, it lets you get multi-lingual names when they are not specified >> in OSM, it allows much more intelligent searches based on types of objects, >> it allows quality controls. The abrakadabra is exactly what one has to do >> when parsing non-standardized data. >> >>> >>> Validation of wikipedia tag values can and IS already done using osm >>> data versus wikipedia-geolocated data extracts/dumps. >>> >>> Sure, it can be via dump parsing, but it is a much more complicated than >> querying. Would you rather use Overpass turbo to do a quick search for >> some weird thing that you noticed, or download and parse the dump? Most >> people would rather do the former. Here is the same thing - you *could* do >> validation via a dump, but that barrier of entry is so high, most people >> wouldn't. With the new OSM+Wikidata tool, which is already getting >> hundreds of thousands requests (!!!) , it is possible to get just the data >> you need, and fix the problems that have been always present, but hidden. >> And all that is possible because of a single tag. >> > ___ talk mailing list talk@openstreetmap.org
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
I think that the underlying issue in wikidata tags is that they are external IDs. Not human readable, they cannot be entered 'by hand' nor verified on the ground. Once you accept them in OSM, you can't really complain about bots. Yves (who still think such UIDs are only needed for the lack of good query tools). Le 26 septembre 2017 19:08:33 GMT+02:00, Yuri Astrakhana écrit : >> >> > p.s. OSM is a community project, not a programmers project, it's >about >> > people, not software :-) >> >> It's both. OSM is first and foremost is a community, but the result >of >our effort is a machine-readable database. We are not creating an >encyclopedia that will be casually flipped through by humans. We >produce >data that gets interpreted by software, so that it can render maps and >be >searchable. For example, if every person uses their own tag names and >ways >to record things, the data will have nearly zero value. We must agree >on >conventions so that software can understand our results - which is >exactly >what we have been doing on wiki and in email channels. Any tag and >value >that cannot be recognized and processed by software is effectively >ignored. > > >> Totally agree. If some script can automatically add new tag >> (wikidata) without any actual WORK needed, then it is pointless, >> anybody can run an auto-update script. > > When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia >> data, they add wikipedia link, not wikidata "stuff". >> > >While sand castles may look nice, they don't last very long. When >ordinary >people add just the Wikipedia article, that link quickly gets stale and >become irrelevant and often incorrect. The wikipedia article titles are >not >stable. They get renamed all the time - there are tens of thousands of >them >in OSM already that I found. Often, community renames wp articles >because >there are more than one meaning, so they create a new article with the >same >name in its place - a disambig page. There is no easy way to analyse >wikipedia links for content - you cannot easily determine if the >wikipedia >article is about a person, a country, or a house, which makes it >impossible >to check for correctness. > >When I spend half an hour of my time researching which WP article is >best >for an object, I do not want that effort to be wasted just because >someone >else puts a disambig page in its place, and I have to redo all my work. > > When data consumers want to get a link to corresponding wikipedia >> article, doing that with wikipedia[:xx] tags is straightforward. >Doing >> the same with wikidata requires additional pointless and time >> consuming abrakadabra. >> > >no, you clearly haven't worked with any data consumers recently. Data >consumers want Wikidata, much more than wikipedia tags - please talk to >them. Wikidata gives you the list of wikipedia articles in all >available >languages, it lets you get multi-lingual names when they are not >specified >in OSM, it allows much more intelligent searches based on types of >objects, >it allows quality controls. The abrakadabra is exactly what one has to >do >when parsing non-standardized data. > >> >> Validation of wikipedia tag values can and IS already done using >osm >> data versus wikipedia-geolocated data extracts/dumps. >> >> Sure, it can be via dump parsing, but it is a much more complicated >than >querying. Would you rather use Overpass turbo to do a quick search for >some weird thing that you noticed, or download and parse the dump? >Most >people would rather do the former. Here is the same thing - you *could* >do >validation via a dump, but that barrier of entry is so high, most >people >wouldn't. With the new OSM+Wikidata tool, which is already getting >hundreds of thousands requests (!!!) , it is possible to get just the >data >you need, and fix the problems that have been always present, but >hidden. >And all that is possible because of a single tag. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> > > p.s. OSM is a community project, not a programmers project, it's about > > people, not software :-) > > It's both. OSM is first and foremost is a community, but the result of our effort is a machine-readable database. We are not creating an encyclopedia that will be casually flipped through by humans. We produce data that gets interpreted by software, so that it can render maps and be searchable. For example, if every person uses their own tag names and ways to record things, the data will have nearly zero value. We must agree on conventions so that software can understand our results - which is exactly what we have been doing on wiki and in email channels. Any tag and value that cannot be recognized and processed by software is effectively ignored. > Totally agree. If some script can automatically add new tag > (wikidata) without any actual WORK needed, then it is pointless, > anybody can run an auto-update script. When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia > data, they add wikipedia link, not wikidata "stuff". > While sand castles may look nice, they don't last very long. When ordinary people add just the Wikipedia article, that link quickly gets stale and become irrelevant and often incorrect. The wikipedia article titles are not stable. They get renamed all the time - there are tens of thousands of them in OSM already that I found. Often, community renames wp articles because there are more than one meaning, so they create a new article with the same name in its place - a disambig page. There is no easy way to analyse wikipedia links for content - you cannot easily determine if the wikipedia article is about a person, a country, or a house, which makes it impossible to check for correctness. When I spend half an hour of my time researching which WP article is best for an object, I do not want that effort to be wasted just because someone else puts a disambig page in its place, and I have to redo all my work. When data consumers want to get a link to corresponding wikipedia > article, doing that with wikipedia[:xx] tags is straightforward. Doing > the same with wikidata requires additional pointless and time > consuming abrakadabra. > no, you clearly haven't worked with any data consumers recently. Data consumers want Wikidata, much more than wikipedia tags - please talk to them. Wikidata gives you the list of wikipedia articles in all available languages, it lets you get multi-lingual names when they are not specified in OSM, it allows much more intelligent searches based on types of objects, it allows quality controls. The abrakadabra is exactly what one has to do when parsing non-standardized data. > > Validation of wikipedia tag values can and IS already done using osm > data versus wikipedia-geolocated data extracts/dumps. > > Sure, it can be via dump parsing, but it is a much more complicated than querying. Would you rather use Overpass turbo to do a quick search for some weird thing that you noticed, or download and parse the dump? Most people would rather do the former. Here is the same thing - you *could* do validation via a dump, but that barrier of entry is so high, most people wouldn't. With the new OSM+Wikidata tool, which is already getting hundreds of thousands requests (!!!) , it is possible to get just the data you need, and fix the problems that have been always present, but hidden. And all that is possible because of a single tag. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> p.s. OSM is a community project, not a programmers project, it's about > people, not software :-) Totally agree. If some script can automatically add new tag (wikidata) without any actual WORK needed, then it is pointless, anybody can run an auto-update script. When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia data, they add wikipedia link, not wikidata "stuff". When data consumers want to get a link to corresponding wikipedia article, doing that with wikipedia[:xx] tags is straightforward. Doing the same with wikidata requires additional pointless and time consuming abrakadabra. Validation of wikipedia tag values can and IS already done using osm data versus wikipedia-geolocated data extracts/dumps. -- Tomas ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Seems, that I was mistaken: https://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_conduct states: Either talk (a general purpose mailing list) or if your edit affects only one country or territory then the national-language mailing lists, forums, or other standard communication methods for the territory affected by the change or ... This seems odd to me, as not all communities affected by a mechanical edit are represented in talk. So this means that the country in which most edits will take place do not have to be consulted ? regards m. p.s. OSM is a community project, not a programmers project, it's about people, not software :-) On Tue, Sep 26, 2017 at 10:02 AM, Andy Mabbettwrote: > On 26 September 2017 at 04:14, Marc Gemis wrote: > >> I believe the mechanical edit polity demands that you discuss with the >> *local* community. That means if your edit modifies items in e.g. >> Mexico, Belgium and Japan, you have to discuss your edit with the >> communities in Mexico, Belgium and Japan. > > That is clearly impractical for a task of this nature and, if true, > would be an effective prohibition on any such beneficial editing. > > -- > Andy Mabbett > @pigsonthewing > http://pigsonthewing.org.uk > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Tuesday 26 September 2017, Yuri Astrakhan wrote: > Since this thread had not received any new discussion in the past 4 > days, I assumed all points were answered and proceeded as planned, > per mechanical edit policy. That is always a bad idea. For example in this thread i made the following comment: > > Christoph, a valid point. Yet the duplicate would allow finding > > many of these errors, rather than leaving wp-only to go bad due to > > changing nature of the WP articles. > > Actually no - you can find the errors just as well without adding the > wikidata tags to OSM as after doing so. You did not react to that so why would it make sense for you to assume this issue has been resolved - just by ignoring it? > Per previous description, the existing data is already bad, and I am > simply making it possible to identify it, after discussing it on this > thread. Sorry but this is utter nonsense. Importing data into OSM is never required for either fixing errors in OSM or in the imported data. If you do an import you need to make sure the results are at least on the same level of quality as they would be based on competent manual mapping. You need to ensure that in data preparation and not after doing the import. And the presence of pre-existing bad data (to a significant part already produced through under-the-radar mechanical edits by the way) is no excuse for adding more bad data. > Andy, Wikidata ID is not correct or incorrect -- [...] Then it is non-verifiable data and does not belong in the OSM database at all. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 26 September 2017 at 04:14, Marc Gemiswrote: > I believe the mechanical edit polity demands that you discuss with the > *local* community. That means if your edit modifies items in e.g. > Mexico, Belgium and Japan, you have to discuss your edit with the > communities in Mexico, Belgium and Japan. That is clearly impractical for a task of this nature and, if true, would be an effective prohibition on any such beneficial editing. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 26 September 2017 at 03:40, Yuri Astrakhanwrote: > I have been blocked by Andy Townsend with the following message. > I believe Andy is acting in best interest of the project, yet might have > missed or misread this discussion. I agree with Yuri. This series of edits will, once complete, leave OSM in a much better state than before; with invalid or outdated Wikipedia tags identified and replaced (or removed); with the bonus of good Wikidata tags added to them. Yuri should be unblocked, and allowed to complete the job. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Marc, thanks. I was under the assumption that talk is the global community - as it is the most generic in the list, unlike talk-us and talk-us-newyork. Does it meany that any global proposal would require talking to hundreds of communities independently, making it impossible to coordinate, because comments in one community would not be visible to other communities? Is there any kind of ambassadorial program? Also, does it mean that talk-us doesn't decide anything because there is a talk-us-newyork? In this specific case, adding wikidata seemed like a long overdue task, something that is already happening automatically by the unmonitored iD feature. Btw, I looked at the descriptions at https://lists.openstreetmap.org/listinfo On Mon, Sep 25, 2017 at 11:14 PM, Marc Gemiswrote: > > moving it here. I believe I acted in good faith according to the > mechanical > > edit policy - discussed with the community, and proceeded. > > I believe the mechanical edit polity demands that you discuss with the > *local* community. That means if your edit modifies items in e.g. > Mexico, Belgium and Japan, you have to discuss your edit with the > communities in Mexico, Belgium and Japan. This might also mean that > you have to discuss it via Telegram, Facebook, email, IRC, etc. > depending on where that local community is. > > The talk mailing list is not sufficient. > > regards > > m. > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> moving it here. I believe I acted in good faith according to the mechanical > edit policy - discussed with the community, and proceeded. I believe the mechanical edit polity demands that you discuss with the *local* community. That means if your edit modifies items in e.g. Mexico, Belgium and Japan, you have to discuss your edit with the communities in Mexico, Belgium and Japan. This might also mean that you have to discuss it via Telegram, Facebook, email, IRC, etc. depending on where that local community is. The talk mailing list is not sufficient. regards m. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Since this thread had not received any new discussion in the past 4 days, I assumed all points were answered and proceeded as planned, per mechanical edit policy. Yet, after I have added all the nodes and moved on to relations, I have been blocked by Andy Townsend with the following message. I believe Andy is acting in best interest of the project, yet might have missed or misread this discussion. Also, the block is such that I am no longer able to even reply on the changesets to the raised questions, so moving it here. I believe I acted in good faith according to the mechanical edit policy - discussed with the community, and proceeded. A few interesting semi-relevant statistics so far: the number of discovered links to disambig pages is now back to over 800, even without 100k+ untaged ways. And there are almost 38,000 osm objects where wikipedia tag does not correspond with wikidata tag. The number is very high, but fixing them should be semi-automated, as most of them are redirects. TBD. Here's Andy's message, with my inlined replies. I think that almost all of the raised points have been raised and answered in our previous discussion, but I feel it is my responsibility to present them again. You're conducting an import of known bad data (your own changeset comments > say "Further cleanup will be done using..."). > Per previous description, the existing data is already bad, and I am simply making it possible to identify it, after discussing it on this thread. > You are wilfully ignoring the feedback that you're receiving now and have > received in the past. A lot of issues have been raised about the quality of > your edits - see > http://resultmaps.neis-one.org/osm-discussion-comments?uid=339581 . In > many cases you seem to agree that you're adding rubbish, and yet you > continue. > You seem to be suggesting (in > https://lists.openstreetmap.org/pipermail/talk/2017-September/078767.html > ) that "the community" clean up your mess. This is not the way that > OpenStreetMap works - if an individual is adding data to it (especially > large quantities of data) then it is their responsibility to ensure that > the data that they are adding is valid, or at least as valid as the data > that is already there. > Again, no, I am identifying rubbish, not introducing it, and I am very actively replying to every comment I receive. This is not "my data" - the data is already in OSM in the form of the incorrect wikipedia tags. This action is identical to what iD editor does - it *automatically* adds corresponding wikidata ID, without any additional checks, and without many users even being aware of it. The way to solve the quality of this data is to analyze it with the OSM+Wikidata tool I have built, to see the mismatches. Since there are tens (hundreds?) of thousands of issues already in the database, it is clearly impossible to fix it by one person. The available choices are: me doing it by hand, and fixing a handful, or make it possible to find problems, so everyone can fix them. (per Andy Mabbett explanation) Please go back and reread some of your previous replies on > http://resultmaps.neis-one.org/osm-discussion-comments?uid=339581 . > Things like "I will mostly work on high level objects (admin level <= 6)" > suggests that you are at the very least being disingenuous in your dealings > with the OSM community. > This was written a long time ago, before this effort was even started, and before I have built the tools (OSM+Wikidata) to let community find issues. Back then I had to do everything myself, and since it was clearly impossible, I stopped after fixing the wast majority of the uncovered issues by hand. > Please stop this mechanical edit now and instead spend your time > addressing the issues that have been raised. > I believe i have answered this numerous times above and in previous conversations. I cannot address tens of thousands of issues i *find*, I can only help community see them, and do my part in fixing them. Without this effort, all the bad data in the form of incorrect wikipedia tags will still be there, quickly rotting away with every wikipedia page rename. P.S. An interesting point was brought by Andy in the later online chat: > > in the case of https://www.openstreetmap.org/changeset/43749373 the > errors were explicitly introduced by you. The links from OSM to wikipedia > were correct, the thing (probably a bot) creating the wikidata from > wikipedia didn't understand the breadth of what the wikipedia article > represented, and you incorrectly linked from OSM to the wikidata article. > Andy, Wikidata ID is not correct or incorrect -- it is simply a number assigned to a Wikipedia article. That number may have other statements, which themselves may be incorrect. Adding Wikidata ID locks that Wikipedia tag in place, to keep it from going stale - in case that page is renamed, and in case a disambig is created in its place. In some cases, the concept presented in
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Thu, Sep 21, 2017 at 9:22 PM, Martin Koppenhoeferwrote: > This is one of the fields (fundamental to OSM), where wikidata is just a > mess: distinction of geographically localized communities and > administrative territorial entities. > > Just a few examples: > https://www.wikidata.org/wiki/Q123705 neighbourhood is a subclass of > "human settlement" and "community". So far so good, but then it is also > "part of municipality"? > https://www.wikidata.org/wiki/Q2983893 quarter is a subclass of > neightbourhood and administrative territorial entity. And it is an instance > of "designation for an administrative territorial entity". > https://www.wikidata.org/wiki/Q486972 human settlement looks OK at a > glance > https://www.wikidata.org/wiki/Q515 city is a subclass of "human > settlement", "administrative territorial entity" and "political territorial > entity" (are these AND or OR?). > https://www.wikidata.org/wiki/Q3957 town is a subclass of "human > settlement". It is "part of a country". It was also a subclass of political > territorial entity until today [1] > https://www.wikidata.org/wiki/Q532 village is a subclass of "rural > settlement" and part of "rural area". > https://www.wikidata.org/wiki/Q14788575 rural settlement is an instance > of "designation for an administrative territorial entity" and a subclass of > "human settlement" > etc. > > Take the town example: it has been for some years a subclass of political > territorial entity and isn't anymore since today. There are tens of > thousands of objects that are all instances of towns according to wikidata. > With one edit all of them have lost their "political territorial entity" > status. > I wouldn't worry too much about these very generic classes of human settlements or administrative areas. It would be better to focus our attention on the actual Wikidata items on settlements and administrative areas of each country, like Waldhufendorfs in Germany ( https://www.wikidata.org/wiki/Q351190) or comunes in Italy ( https://www.wikidata.org/wiki/Q747074). These national subclasses should then be classified as subclasses of whichever appropriate generic settlements or administrative areas there are. So I doubt that tens of thousands of towns in Wikidata suddenly lost their political territorial entity status. I would think that some of them are still classified as such because of a different path up the ontology tree. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-09-20 19:07 GMT+02:00 Christoph Hormann: > Don't assume such cases are just a freak anomaly - they are not. OSM > and wikidata are two very different projects which developed in very > different contexts. Just another example: For most cities and larger > towns (at least in Germany) there exists an admin_level 6/8 unit with > the same name and most of these seem to have a single wikidata item > while in OSM we have two separate concepts for the populated place > (place=city/town) and the administrative unit (boundary relation with > boundary=administrative + admin_level=6/8). +1 This is one of the fields (fundamental to OSM), where wikidata is just a mess: distinction of geographically localized communities and administrative territorial entities. Just a few examples: https://www.wikidata.org/wiki/Q123705 neighbourhood is a subclass of "human settlement" and "community". So far so good, but then it is also "part of municipality"? https://www.wikidata.org/wiki/Q2983893 quarter is a subclass of neightbourhood and administrative territorial entity. And it is an instance of "designation for an administrative territorial entity". https://www.wikidata.org/wiki/Q486972 human settlement looks OK at a glance https://www.wikidata.org/wiki/Q515 city is a subclass of "human settlement", "administrative territorial entity" and "political territorial entity" (are these AND or OR?). https://www.wikidata.org/wiki/Q3957 town is a subclass of "human settlement". It is "part of a country". It was also a subclass of political territorial entity until today [1] https://www.wikidata.org/wiki/Q532 village is a subclass of "rural settlement" and part of "rural area". https://www.wikidata.org/wiki/Q14788575 rural settlement is an instance of "designation for an administrative territorial entity" and a subclass of "human settlement" etc. Take the town example: it has been for some years a subclass of political territorial entity and isn't anymore since today. There are tens of thousands of objects that are all instances of towns according to wikidata. With one edit all of them have lost their "political territorial entity" status. Cheers, Martin [1] https://www.wikidata.org/w/index.php?title=Q3957=revision=563002649=552128156 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
As for the Faroer Islands (https://www.wikidata.org/wiki/Q4628) I see some contradicting information there.The labels refer to an archipelago, while the "is a" statement refers to an administrative part of Denmark. When an item has only 1 "is a" statement, it is not possible to refer to 2 different concepts. However, it is possible to have to "is a" statements on an item. m. On Wed, Sep 20, 2017 at 6:02 PM, Christoph Hormannwrote: > On Wednesday 20 September 2017, Eugene Alvin Villar wrote: >> >> This is most certainly a wrong modeling in Wikidata. While we can >> just have one well-written and comprehensive Wikipedia article about >> the country/archipelago, in Wikidata, one item should correspond to >> one concept. > > Maybe - but even if that is the case the wikidata concept of a country > or archipelago is not necessarily the same as in OSM. For countries > and archipelagos this might sound strange (an archipelago is an > archipelago, right?) but as you surely know the meaning of tags in OSM > can be quite peculiar in the way it develops over time based on mapping > needs and it would be quite insane if wikidata copied all these > peculiarities in their classification system. > > I don't know a lot about wikidata but as far as i can see every > wikipedia article links to exactly one wikidata item and there are many > geographic wikipedia articles that describe several different concepts > together for which separate OSM features exist. > > -- > Christoph Hormann > http://www.imagico.de/ > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Am 20.09.2017 um 20:55 schrieb Yuri Astrakhan: > While the WMF does not claim any rights in wikidata contents, it > does not make any representations (one way or the other) as to > third party rights in the data. As an illustration: you could dump > all of OSM in to wikidata and the WMF would not need to change or > do anything. > > > But the same works in reverse, doesn't it? Wikidata project, just > like WP and OSM, is user contributable. If a user uploads data that > violates project's license, it should be deleted. And for that reason, > both Wikidata and OSM state the license under which the data is > contributed and shared. If I make an edit to OSM by copying data from > Google, wouldn't that be the same thing? The WMFs doctrine is that data (even more than one item) is not protectable, the wording on the WD edit page is ambiguous and the ToU don't really address the issue at all. Further the WMF is not known for policing wikidata (contrary to the OSMF and OSM) and it is doubtful if it could even be done in any reasonable way. Skipping that lots of WD data was originally derived from WP with its own set of issues. That said, as long as we don't start using wikidata instead of data from OSM contributors, it really is just the WMFs problem. not ours. We really really have better use for brain power than trying to fix the WMFs problems for them. > > >> (CC0), but the reverse depends on if the OSM contributor agreed >> to dedicate their edits to public domain. > There is not really a practical and meaningful way in which an OSM > contributor could do that, outside of facts that they have > surveyed themselves and kept separate. > > > How I hate to diverge from the main topic, but alas... :) This does > sound like a severe problem (that should be taken to a separate > thread) - if I, as a user, set the Public Domain checkbox, my > assumptions are that my contributions are PD. If I trace something > based on some image data, I need to specify that source, otherwise I > am in violation of the source's license. If I did not specify the > source, and I checked the PD box, it can be assumed that I am donating > under PD. If this is not the case, it is a violation of my > contributor's rights - because otherwise my intention is not being > honored (i want other people to be able to use my work unrestricted). > If anyone wants to comment, please start a new thread :) This has really been beaten to death: at best the PD flag can be taken as an indication of sentiment. Fixing it would require re-wording the actual text, going back to 4 million odd users and asking them to reconsider their choice. This however would not address the already mentioned fundamental issues with data prior to such a change (assuming that it would be practical to implement all the technical measures that you are suggesting going forward) and further would still run afoul of the fact that the OSMF doesn't have a mandate, is not even allowed, to distribute contributed data on any other terms than those compatible with the contributor terms. NOT GOING TO HAPPEN (at least not on volunteer time). Note on the side: If we were to undertake anything even remotely on the scale of what the above would imply, it is likely that we would review our current licence instead. However as has been pointed out many times that would not result is us switching to a non-attribution licence (aka CC0 or similar), so it wouldn't really help with wikidata compatibility. > >> Without it, OSM data is licensed under ODbL, and cannot be >> copied. We should make it easier to detect what piece of OSM data >> is in PD. I do like your USB analogy :) About names - you will >> be surprised to discover that MB and other places are actively >> pursuing Wikidata integration because WD tends to have a huge >> names list, possibly bigger than OSM itself? >> >> > That is nice for MB, but problematic in more than one way for OSM. > > Please elaborate, I know of at least one more company that is actively > doing that. Sigh, another side topic :D Very simple: use of wikidata is not declared and not obvious to the end users, errors in wikidata get attributed to OSM but can't be fixed in OSM, well can't be fixed without a lot of technical mumbo-jumbo that you cannot expect non-seasoned hands to know. And even if the user in the end finds out where to fix an issue, they are spending time fixing wikidata, not OSM. It is completely clear that we are in a competitive situation for mind share, money and contributors (more exact: for contributors time) with many other players. Now OSM proper has been loosing out big time on the first point as of late, but luckily hasn't had great requirements on the 2nd (that is why we are still around), but even OSM is not so daft to want a situation in which it actively has to redirect potential OSM contributors to a third party to fix "its" core data. Simon PS:
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> > people fixing WD won’t necessarily check if their fixes work well with > OSM. Maybe we should include versions in our WD tags? > I’ve seen OSM objects linked from WD, are there people monitoring changes > to linked objects? > Yes, that's what the Wikidata+OSM service is for. It allows community to create queries that verify various aspects of OSM objects as related to Wikidata Objects. For example, if Wikidata object changes its "instance-of" to disambig, the query would immediatelly flag corresponding OSM object as having a problematic wiki link. > > I think it’s better to add the WD links slowly, verifying on a one by one > basis that the objects describe the same thing. And having this done for > some time I can tell that quite often WD items are very basic and defined > besides their name only by the content of their WP article links, which in > different languages not always describe the same thing/s. > If you look into the things there’s a lot to fix in both projects, adding > WD tags automatically in one go might help less than people doing it > carefully and fixing the problems on the way. > > iD editor has been doing exactly that for substantial time. Whenever user adds a wikipedia tag, corresponding Wikidata tag is added automatically. I seriously doubt there are any (or any at all) people who check that wikidata ID is correct by hand. Yet the number of errors that were caught by cross-linking the data is very significant. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
sent from a phone > On 20. Sep 2017, at 17:37, Eugene Alvin Villarwrote: > > This is most certainly a wrong modeling in Wikidata. While we can just have > one well-written and comprehensive Wikipedia article about the > country/archipelago, in Wikidata, one item should correspond to one concept. > So there should be separate Wikidata items for the archipelago and the > country. The fact that it isn't like that right now is simply because > Wikidata is an ongoing project, just like OSM. people fixing WD won’t necessarily check if their fixes work well with OSM. Maybe we should include versions in our WD tags? I’ve seen OSM objects linked from WD, are there people monitoring changes to linked objects? I think it’s better to add the WD links slowly, verifying on a one by one basis that the objects describe the same thing. And having this done for some time I can tell that quite often WD items are very basic and defined besides their name only by the content of their WP article links, which in different languages not always describe the same thing/s. If you look into the things there’s a lot to fix in both projects, adding WD tags automatically in one go might help less than people doing it carefully and fixing the problems on the way. cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> > While the WMF does not claim any rights in wikidata contents, it does not > make any representations (one way or the other) as to third party rights in > the data. As an illustration: you could dump all of OSM in to wikidata and > the WMF would not need to change or do anything. > But the same works in reverse, doesn't it? Wikidata project, just like WP and OSM, is user contributable. If a user uploads data that violates project's license, it should be deleted. And for that reason, both Wikidata and OSM state the license under which the data is contributed and shared. If I make an edit to OSM by copying data from Google, wouldn't that be the same thing? > (CC0), but the reverse depends on if the OSM contributor agreed to > dedicate their edits to public domain. > > There is not really a practical and meaningful way in which an OSM > contributor could do that, outside of facts that they have surveyed > themselves and kept separate. > How I hate to diverge from the main topic, but alas... :) This does sound like a severe problem (that should be taken to a separate thread) - if I, as a user, set the Public Domain checkbox, my assumptions are that my contributions are PD. If I trace something based on some image data, I need to specify that source, otherwise I am in violation of the source's license. If I did not specify the source, and I checked the PD box, it can be assumed that I am donating under PD. If this is not the case, it is a violation of my contributor's rights - because otherwise my intention is not being honored (i want other people to be able to use my work unrestricted). If anyone wants to comment, please start a new thread :) > > Without it, OSM data is licensed under ODbL, and cannot be copied. We > should make it easier to detect what piece of OSM data is in PD. I do like > your USB analogy :) About names - you will be surprised to discover that MB > and other places are actively pursuing Wikidata integration because WD > tends to have a huge names list, possibly bigger than OSM itself? > > > That is nice for MB, but problematic in more than one way for OSM. > Please elaborate, I know of at least one more company that is actively doing that. Sigh, another side topic :D On Wed, Sep 20, 2017 at 1:58 PM, Simon Poolewrote: > [turning on broken record mode :-)] > On 20.09.2017 17:54, Yuri Astrakhan wrote: > > > > * Oleksiy, OSM can use any data from Wikidata because of the public domain > dedication > > While the WMF does not claim any rights in wikidata contents, it does not > make any representations (one way or the other) as to third party rights in > the data. As an illustration: you could dump all of OSM in to wikidata and > the WMF would not need to change or do anything. > > (CC0), but the reverse depends on if the OSM contributor agreed to > dedicate their edits to public domain. > > There is not really a practical and meaningful way in which an OSM > contributor could do that, outside of facts that they have surveyed > themselves and kept separate. > > Without it, OSM data is licensed under ODbL, and cannot be copied. We > should make it easier to detect what piece of OSM data is in PD. I do like > your USB analogy :) About names - you will be surprised to discover that MB > and other places are actively pursuing Wikidata integration because WD > tends to have a huge names list, possibly bigger than OSM itself? > > > That is nice for MB, but problematic in more than one way for OSM. > > Simon > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
[turning on broken record mode :-)] On 20.09.2017 17:54, Yuri Astrakhan wrote: > > > * Oleksiy, OSM can use any data from Wikidata because of the public > domain dedication While the WMF does not claim any rights in wikidata contents, it does not make any representations (one way or the other) as to third party rights in the data. As an illustration: you could dump all of OSM in to wikidata and the WMF would not need to change or do anything. > (CC0), but the reverse depends on if the OSM contributor agreed to > dedicate their edits to public domain. There is not really a practical and meaningful way in which an OSM contributor could do that, outside of facts that they have surveyed themselves and kept separate. > Without it, OSM data is licensed under ODbL, and cannot be copied. We > should make it easier to detect what piece of OSM data is in PD. I do > like your USB analogy :) About names - you will be surprised to > discover that MB and other places are actively pursuing Wikidata > integration because WD tends to have a huge names list, possibly > bigger than OSM itself? > > That is nice for MB, but problematic in more than one way for OSM. Simon ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 20.09.17 18:35, Yuri Astrakhan wrote: One other thing: lets not build walls between different projects. I know its in a human nature to do that, but lets not. In Wikipedia, every language is also a separate project, and there I also saw a lot of "this is not how we do things around here". [...] Here is how the same idea was expressed in a classical poem, probably one of the best written in English: https://youtu.be/DJBLG0FvFMM The poem ends in lines: ‘Men work together,’ I told him from the heart, ‘Whether they work together or apart.’ brgds O. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
2017-09-20 19:07 GMT+02:00 Christoph Hormann: > > > Don't assume such cases are just a freak anomaly - they are not. OSM > and wikidata are two very different projects which developed in very > different contexts. Just another example: For most cities and larger > towns (at least in Germany) there exists an admin_level 6/8 unit with > the same name and most of these seem to have a single wikidata item > while in OSM we have two separate concepts for the populated place > (place=city/town) and the administrative unit (boundary relation with > boundary=administrative + admin_level=6/8). > > Hello Christoph, this specific case was pointed out some time ago also on the italian mailing list. You're right on the issue, but on Wikidata recently someone created the entity for the proper cities in Italy. Here's the sparql query (all the objects which are marked capital of an italian municipality) http://tinyurl.com/yb2cjbae Not all the municipalities have this property (6k objects instead of 8k) > -- > Christoph Hormann > http://www.imagico.de/ > Ciao, Stefano > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wednesday 20 September 2017, Yuri Astrakhan wrote: > > https://www.wikidata.org/wiki/Wikidata:WikiProject_Municipalities_of_ >Germany > > I think if you suggest it there, they will be happy to add it, > allowing OSM objects to be properly tagged. Or just contribute there > :) But i don't think there is anything wrong with how wikidata represents things - for Hamburg wikidata for example has a single item: https://www.wikidata.org/wiki/Q1055 which represents both the administrative unit and the populated place - which is perfectly fine. And OSM does differentiate between them which is also fine. Why should i attempt to change their data model to be identical to that of OSM (which would be a hopeless endeavor anyway because how things are represented in OSM is constantly changing)? -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Also, there is a general country subdivision project with plenty of information and current status. I'm pretty sure OSM community has a lot of good info to share: https://www.wikidata.org/wiki/Wikidata:WikiProject_Country_subdivision On Wed, Sep 20, 2017 at 1:28 PM, Yuri Astrakhanwrote: > Don't assume such cases are just a freak anomaly - they are not. OSM >> and wikidata are two very different projects which developed in very >> different contexts. Just another example: For most cities and larger >> towns (at least in Germany) there exists an admin_level 6/8 unit with >> the same name and most of these seem to have a single wikidata item >> while in OSM we have two separate concepts for the populated place >> (place=city/town) and the administrative unit (boundary relation with >> boundary=administrative + admin_level=6/8). >> >> Christoph, Wikidata community has a project > > https://www.wikidata.org/wiki/Wikidata:WikiProject_ > Municipalities_of_Germany > > I think if you suggest it there, they will be happy to add it, allowing > OSM objects to be properly tagged. Or just contribute there :) > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> > Don't assume such cases are just a freak anomaly - they are not. OSM > and wikidata are two very different projects which developed in very > different contexts. Just another example: For most cities and larger > towns (at least in Germany) there exists an admin_level 6/8 unit with > the same name and most of these seem to have a single wikidata item > while in OSM we have two separate concepts for the populated place > (place=city/town) and the administrative unit (boundary relation with > boundary=administrative + admin_level=6/8). > > Christoph, Wikidata community has a project https://www.wikidata.org/wiki/Wikidata:WikiProject_Municipalities_of_Germany I think if you suggest it there, they will be happy to add it, allowing OSM objects to be properly tagged. Or just contribute there :) ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wednesday 20 September 2017, Yuri Astrakhan wrote: > > Christoph, a valid point. Yet the duplicate would allow finding many > of these errors, rather than leaving wp-only to go bad due to > changing nature of the WP articles. Actually no - you can find the errors just as well without adding the wikidata tags to OSM as after doing so. The perpetuation of errors is one of the primary reasons why mechanical edits are often not considered favorably in OSM. > As for sameness argument - lets > try to work on them on a case-by-case basis. I don't really want to argue this here - as said i have no objections against having wikipedia/wikidata tags as references for 'related features' but many treat these references under the assumption that they indicate identical real world concepts exist on both ends - or even worth: they might think a lack of identity is an indication for a factual error in the data on one side. I wanted to point out that this is a fundamentally incorrect assumption. Don't assume such cases are just a freak anomaly - they are not. OSM and wikidata are two very different projects which developed in very different contexts. Just another example: For most cities and larger towns (at least in Germany) there exists an admin_level 6/8 unit with the same name and most of these seem to have a single wikidata item while in OSM we have two separate concepts for the populated place (place=city/town) and the administrative unit (boundary relation with boundary=administrative + admin_level=6/8). -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
One other thing: lets not build walls between different projects. I know its in a human nature to do that, but lets not. In Wikipedia, every language is also a separate project, and there I also saw a lot of "this is not how we do things around here". Each project is ran by people. Most people contribute to more than one project, so lets not say "they do X, but we do Y", because often "they and us" is the same person. Obviously some rules differ, and we should respect that, but (I hope) most of us dedicate our volunteer time because we believe in the general principal: making knowledge available to everyone freely. OSM concentrates on geographical knowledge. Wikidata - on classification. Wikipedia - on descriptions. All three, as a sum, can be much greater than each one separately. Lets keep that in mind when we think how we can coexist better, and how to reduce the overlap. Keeping duplicates in sync is always harder than to let the tools do their data merging work if needed. On Wed, Sep 20, 2017 at 12:18 PM, Yuri Astrakhanwrote: > Tobias, agree 100%, thanks. > > On Wed, Sep 20, 2017 at 12:14 PM, Tobias Knerr > wrote: > >> On 20.09.2017 17:02, Christoph Hormann wrote: >> > It is best to regard the wikidata and wikipedia tags in OSM as 'related >> > features' rather than identical objects. >> >> We shouldn't dilute the definition of the key because some incorrect >> links exist in the database. If there's no 1:1 relationship between OSM >> element and Wikidata item, and this cannot be fixed by editing Wikidata, >> then no wikidata tag should be added. >> >> > These provide useful sources >> > to research additional information (in particular wikipedia articles >> > often link to additional sources) >> >> Sure, but the wikidata key is for "the Wikidata item about the feature", >> not any related content that may be interesting. >> >> ___ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk >> > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Tobias, agree 100%, thanks. On Wed, Sep 20, 2017 at 12:14 PM, Tobias Knerrwrote: > On 20.09.2017 17:02, Christoph Hormann wrote: > > It is best to regard the wikidata and wikipedia tags in OSM as 'related > > features' rather than identical objects. > > We shouldn't dilute the definition of the key because some incorrect > links exist in the database. If there's no 1:1 relationship between OSM > element and Wikidata item, and this cannot be fixed by editing Wikidata, > then no wikidata tag should be added. > > > These provide useful sources > > to research additional information (in particular wikipedia articles > > often link to additional sources) > > Sure, but the wikidata key is for "the Wikidata item about the feature", > not any related content that may be interesting. > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 20.09.2017 17:02, Christoph Hormann wrote: > It is best to regard the wikidata and wikipedia tags in OSM as 'related > features' rather than identical objects. We shouldn't dilute the definition of the key because some incorrect links exist in the database. If there's no 1:1 relationship between OSM element and Wikidata item, and this cannot be fixed by editing Wikidata, then no wikidata tag should be added. > These provide useful sources > to research additional information (in particular wikipedia articles > often link to additional sources) Sure, but the wikidata key is for "the Wikidata item about the feature", not any related content that may be interesting. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
> > What will inevitably happen if you automatically add wikidata tags is > that existing errors in either OSM (in form of incorrect wikipedia > tags) or in wikidata (in form of incorrect connections to wikipedia > articles) will get duplicated. > Christoph, a valid point. Yet the duplicate would allow finding many of these errors, rather than leaving wp-only to go bad due to changing nature of the WP articles. As for sameness argument - lets try to work on them on a case-by-case basis. The vast majority of concepts are "good enough" - if a park is tagged with the wikidata id for that park, and someone extends it to add a few more trees, its not a big problem. If that edit combines two parks into one, eventually it would get fixed, with two parks being created. And no, we won't be able to solve every edge case, but we will solve it for the vast majority of them. After all, a map is an approximation of the real world, not a perfect replica. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wednesday 20 September 2017, Eugene Alvin Villar wrote: > > This is most certainly a wrong modeling in Wikidata. While we can > just have one well-written and comprehensive Wikipedia article about > the country/archipelago, in Wikidata, one item should correspond to > one concept. Maybe - but even if that is the case the wikidata concept of a country or archipelago is not necessarily the same as in OSM. For countries and archipelagos this might sound strange (an archipelago is an archipelago, right?) but as you surely know the meaning of tags in OSM can be quite peculiar in the way it develops over time based on mapping needs and it would be quite insane if wikidata copied all these peculiarities in their classification system. I don't know a lot about wikidata but as far as i can see every wikipedia article links to exactly one wikidata item and there are many geographic wikipedia articles that describe several different concepts together for which separate OSM features exist. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wednesday 20 September 2017, Frederik Ramm wrote: > > Does that mean that the original plan of "fetching" (dare I say > importing) 200k Wikidata links through automated connections from > existing Wikipedia links is (a) dangerous because you can easily > obtain a reference to something totally different, or (b) no problem > because it is *to be expected* that an object's Wikipedia and > Wikidata links point to different things an hence the import wouldn't > introduce "errors" per se? It all depends on how you use the data. I think adding the wikidata tags is fine *because* i regard them as simple references to related features but if you'd insist on the idea that the OSM feature and the wikidata item refer to the same real world feature then inferring such an identity from an existing wikipedia tag is even more problematic - because the wikipedia tag was almost certainly not originally verified to refer to exactly the same concept as the OSM object. Also keep in mind that both the OSM features and the wikidata items evolve over time and not every edit made in OSM (like extending the area of a forest polygon to include some additional tree covered area) is necessarily verified to still justify having the wikidata reference. What will inevitably happen if you automatically add wikidata tags is that existing errors in either OSM (in form of incorrect wikipedia tags) or in wikidata (in form of incorrect connections to wikipedia articles) will get duplicated. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Such an awesome discussion, thanks! * https://www.wikidata.org/wiki/Special:GoToLinkedPage can already be used to open a Wikipedia page when you only have a Wikidata ID. It even accepts a list of wiki sites. For example, this link automatically opens the wiki page for Q3669 in the first available language ("pt" in this case) https://www.wikidata.org/wiki/Special:GoToLinkedPage?itemid=Q3669=enwiki,ptwiki * Sarah, thanks for the heads up about Nominatim using Wikipedia tags. I recently added page popularity (pageviews) to the OSM+Wikidata service. Another metric is the number of Wikipedia articles in different languages per topic (sitelinks count). Together, they can be used to calculate relative weights. * I am a bit radical, but not enough to propose we get rid wikipedia tags just yet. They sometimes provide a good indication of the original intent. Once Wikidata is used in all the tooling, we may revisit, but not until then. But yes, wikipedia tags are very unstable, especially when articles get renamed because multiple places have identical names, thus creating a link to disambig. So in general, they often go stale and become less useful without any indication. * Oleksiy, OSM can use any data from Wikidata because of the public domain dedication (CC0), but the reverse depends on if the OSM contributor agreed to dedicate their edits to public domain. Without it, OSM data is licensed under ODbL, and cannot be copied. We should make it easier to detect what piece of OSM data is in PD. I do like your USB analogy :) About names - you will be surprised to discover that MB and other places are actively pursuing Wikidata integration because WD tends to have a huge names list, possibly bigger than OSM itself? * Christoph, a very valid point in general. Do you have any statistics on how often multiple meanings per osm object is a problem? In my experience, this is very rare, but hard to say without numbers. For the case of the island being both a country and a land feature, I think it would benefit OSM to actually have two objects with the same geometry - e.g. two relations containing the same way(s). One relation would treat it as an admin boundary, with all the related tags, the other - as a land feature. Data consumers would treat them separately. Conflating tags related to both concepts into one object is not very good. In a more general terms, you usually have three cases: -- 1:1 (most common imo) -- one osm obj being a part of larger page (e.g. a list of churches). I don't think wikidata/wikipedia tag is appropriate in this case, as that page is not about this specific object, but about a class of similar objects. We could use listed-on:wp, or partof:wp, or some other tag. -- Your case - multiple concepts for the same object. Use either a semicolon separated list of wd ids, or (better) - create multiple relations to describe multiple concepts. * Frederik, that bit of a small personal attack is uncalled for. I exposed a lot of existing bad data, not added it. And I created complex tooling to help everyone resolve it as a community, rather than try to tackle all of it by myself. A system for fixing problems is always better than one person doing it by hand, and later retiring because the challenge is too great. Also, corresponding wikidata tag is not a bad data - it is simply a copy of the existing Wikipedia tag, making it easier for tools and humans to find and fix. As for your last email - fetching *corresponding* wikidata items is not an error - its a duplicate of an existing information. That information might be incomplete, but that's a separate issue. * Lester, I'm not sure I understood your Douglas Adams example, PM me and lets try to figure it out. It might has to do with ranking of each statement See also: Feature request for any lang fallback: https://phabricator.wikimedia.org/T176321 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wed, Sep 20, 2017 at 11:02 PM, Christoph Hormannwrote: > Simple example: The Faroe Islands are both a country: > > http://www.openstreetmap.org/relation/52939 > > and an archipelago: > > http://www.openstreetmap.org/relation/3067431 > > in OSM which are represented as separate features obviously. Both > reference the same wikidata item [...] > This is most certainly a wrong modeling in Wikidata. While we can just have one well-written and comprehensive Wikipedia article about the country/archipelago, in Wikidata, one item should correspond to one concept. So there should be separate Wikidata items for the archipelago and the country. The fact that it isn't like that right now is simply because Wikidata is an ongoing project, just like OSM. As an example, we have the island of Bali ( https://www.wikidata.org/wiki/Q4648) versus the province of Bali ( https://www.wikidata.org/wiki/Q3125978), though there is only just one English Wikipedia article covering both concepts: https://en.wikipedia.org/wiki/Bali ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Hi, On 20.09.2017 17:02, Christoph Hormann wrote: > It is best to regard the wikidata and wikipedia tags in OSM as 'related > features' rather than identical objects. These provide useful sources > to research additional information (in particular wikipedia articles > often link to additional sources) but you should never try to fix or > add something in OSM - be that a name tag or coordinates - based purely > on the assumption that the wikidata object referenced via tag is the > same as the OSM feature. Does that mean that the original plan of "fetching" (dare I say importing) 200k Wikidata links through automated connections from existing Wikipedia links is (a) dangerous because you can easily obtain a reference to something totally different, or (b) no problem because it is *to be expected* that an object's Wikipedia and Wikidata links point to different things an hence the import wouldn't introduce "errors" per se? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On Wednesday 20 September 2017, Frederik Ramm wrote: > > If the Wikidata ID can be fetched automatically based on the > Wikipedia tag, can we delete the Wikipedia tags from everything that > has Wikidata afterwards because it is redundant? This idea stems from the widespread view that a wikipedia article, a wikidata item and an OSM feature refer to the same real world entity just because they reference each other. This is not generally the case - and it can't be since what makes something a certain feature with certain tags in OSM differs fundamentally from what constitutes a certain class of objects in wikidata or what a certain wikipedia article describes. Simple example: The Faroe Islands are both a country: http://www.openstreetmap.org/relation/52939 and an archipelago: http://www.openstreetmap.org/relation/3067431 in OSM which are represented as separate features obviously. Both reference the same wikidata item and all the blind automated name adding activities based on wikidata will not differentiate between names that apply to the archipelago and names of the country (which are not necessarily always the same in all languages). It is best to regard the wikidata and wikipedia tags in OSM as 'related features' rather than identical objects. These provide useful sources to research additional information (in particular wikipedia articles often link to additional sources) but you should never try to fix or add something in OSM - be that a name tag or coordinates - based purely on the assumption that the wikidata object referenced via tag is the same as the OSM feature. -- Christoph Hormann http://www.imagico.de/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
Am 20.09.2017 um 15:02 schrieb Oleksiy Muzalyev: > I apologize for ignorance, but what is MB? I also did not understand > - what is more neutral? Adding the wikidata tag or wikipedia & > wikimedia tag? MB==Mapbox. For now retaining the WP link seems to be in our best interest as otherwise it requires a data consumer to use wikidata to get a useful reference to wikipedia (see for example the use in Nominatim as Sarah has pointed out). Naturally from a pure CS pov that is naturally unnecessary redundancy, but we have that all over the place. Simon > > I noticed during disambiguation error corrections that some of these > errors appear when a Wikipedia article was renamed, but only in the > Wikipedia, and not in the OSM. Wikipedia articles are being renamed > quite often, for example when a town was renamed. > > As far as I know wikidata items are not renamed. Wikidata item tag is > only 7 characters, while wikipedia or wikimedia link could be dozens > of characters. So it takes less space in the database, it is easy to > export, etc. > > Best regards, > Oleksiy > > On 9/20/2017 2:31 PM, Simon Poole wrote: >> You raise an important point. >> >> I've commented before, years back in the mean time, on the push to move >> information out of OSM in to a third party product over which have no >> control and which, if we are not careful, could impact the value of what >> we in OSM are doing and distributing (just see MBs use of wikidata in >> lieu of OSM place names). It would further be very naive to not see the >> competitive angle of what is happening here. >> >> On the other hand lots of the wikidata related additions would seem to >> be fairly neutral (WD references additionally to WP links and so on). >> >> Simon > signature.asc Description: OpenPGP digital signature ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
I apologize for ignorance, but what is MB? I also did not understand - what is more neutral? Adding the wikidata tag or wikipedia & wikimedia tag? I noticed during disambiguation error corrections that some of these errors appear when a Wikipedia article was renamed, but only in the Wikipedia, and not in the OSM. Wikipedia articles are being renamed quite often, for example when a town was renamed. As far as I know wikidata items are not renamed. Wikidata item tag is only 7 characters, while wikipedia or wikimedia link could be dozens of characters. So it takes less space in the database, it is easy to export, etc. Best regards, Oleksiy On 9/20/2017 2:31 PM, Simon Poole wrote: You raise an important point. I've commented before, years back in the mean time, on the push to move information out of OSM in to a third party product over which have no control and which, if we are not careful, could impact the value of what we in OSM are doing and distributing (just see MBs use of wikidata in lieu of OSM place names). It would further be very naive to not see the competitive angle of what is happening here. On the other hand lots of the wikidata related additions would seem to be fairly neutral (WD references additionally to WP links and so on). Simon ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 20 September 2017 at 12:50, JBwrote: > Le 20/09/2017 à 13:05, Oleksiy Muzalyev a écrit : >> It would give a boost to the Wikidata project. > > Am I really reading from an OSM mailing list here? Yes. I read that as "the project at https://wiki.openstreetmap.org/wiki/Wikidata " and not "the project at https://www.wikidata.org/ ". ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 20 September 2017 at 00:56, Frederik Rammwrote: > On 09/19/2017 10:03 PM, Yuri Astrakhan wrote: >> I would like to auto-add all the >> corresponding wikidata based on wikipedia, for all remaining objects, >> using JOSM's "Fetch Wikidata IDs". > > If the Wikidata ID can be fetched automatically based on the Wikipedia > tag, can we delete the Wikipedia tags from everything that has Wikidata > afterwards because it is redundant? Technically, yes, but some in the community have objected to that being done. >> This way, we will be able to quickly find all the objects that are >> problematic with the Wikidata+OSM service. > > Adding problematic data to OSM in order to have it fixed is never a good > idea. That is not what is proposed. It will highlight bad data (Wikipedia links) that is *already* in OSM. >> For example, thanks to the >> community, we already fixed over 600 incorrect links to wiki >> disambiguations pages, > > But this would have been possible without importing the data first? Not as easily, nor as quickly. >> We will be >> able to fix when things are tagged as people (e.g. wikidata -> person, >> instead of subject:wikidata -> person), > > I don't understand; you say that you want to add the wikidata tag to OSM > and only afterwards can you find problems like this? Again, this is where OSM already has a bad Wikipedia link. >> find location errors (e.g. >> wikidata and OSM point to very different locations, implying that its an >> incorrect link). > > Again, OpenStreetMap is not a workbench for importing and then fixing > non-OSM data (even if it may look convenient). Please build a QA process > based on the un-imported data and import it once you have fixed the > problems. And again; this is for bad data that is already in OSM. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
On 19 September 2017 at 23:31, Yuri Astrakhanwrote: >> The commonest error I have found is wikidata=Qnnn instead of >> brand:wikidata=Qnnn for franchises like McDonalds and petrol stations. >> > Andy, I agree - there are many ones like that, all around the globe. I know > that in Israel, @SwiftFast uses a template to keep them in sync for gas > stations and ATMs, but we need a more generic solution. They should be easy to find by looking for items with the same QID, but geographically remote from each other. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk