subject:"\\\[OSM\\\-talk\\\] Adding wikidata tags to the remaining objects with only wikipedia tag"

>>   So you have two tables of same structure. Voila. You can compare
>> anything (title, coordinates), in any direction with some
>> approximation if needed etc. No OSM wikidata involved at all.
>
> Thomas, this will not work. Matching wikidata & osm by coordinates is
> useless, because the coordinates differ too much -- see the hard data proof
> <...>

  I will repeat that this is not something which COULD be done, this
comparison is something, what IS ACTUALLY DONE and has been done for
years.

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread dkiselev

You can compareanything (title, coordinates), in any direction with someapproximation if needed etc.  That's the root of an evil. That comparison have to be done manually. I really don't understand why wikidata needs to be added. It needs to be added to avoid manual fixes of wikipedia links,because wikipedia's articles names aren't constant. That looks like inventing sphisticated comparison procedurewhich has to be done manualy only not to have wikidata tags. What's the point in not having wikidata?  02.10.2017, 05:15, "john whelan" :Rather than fill OSM up with automated edits that have not even been discussed with the local community can we think more about functionality? Since an OSM object has  lat and long value and it appears that wiki whatever also has one the entries can be linked. "This gives you a very simple table with: lat/lonThat/page_title.  No parsing or anything else involved.  You then take data from OSM - lat/lon/wikipedia.tag  So you have two tables of same structure. Voila. You can compareanything (title, coordinates), in any direction with someapproximation if needed etc. No OSM wikidata involved at all." I really don't understand why wikidata needs to be added.  Note the word need, I'm missing the requirement somehow that overides following normal OSM practices. Cheerio John On 1 Oct 2017 7:19 pm,  wrote:Hi everybody.We already accepted wikipedia links keep in mind that wiki article isn't the same abstraction as OSM object.And the way we make a reference on wiki articles differs over time.It was a link, it was an article name, it was a name with language prefix.Wikidata id is a way to make a refererence onto wikipedia articlewhich wiki community states to be the right and consistent one.Why don't we just accept the recomended way of referencing wiki articles?That doesn't make wikipedia articles more or less verifable on the ground.But that makes the link persistent agains the changes in wikipedia.01.10.2017, 16:38, "Christoph Hormann" :> On Sunday 01 October 2017, Marc Gemis wrote:>>  If I noticed an inception date on a information sign next to a>>  building, is this original research or a secondary source ?>> The date on the sign is verifiable as a signed date, not necessarily as> a date connected to the building. Historic information like dates from> before the timeframe witnessed by today's mappers is something that is> problematic in OSM in general.>> As any historian will be able to tell you history as a science is not> about analyzing events of the past in the same way you analyze present> day observations in empiric science disciplines but about analyzing and> interpreting sources - in this way it resembles the approach of> Wikipedia more than that of OSM.>>>  If a Wikidata contributor/Wikipedian researches websites like Orbis,>>  Onroerend erfgoed, etc and finds different inception dates and>>  records them all, is it bad that s/he uses secondary sources instead>>  of the information sign on the ground ?>> I don't want to judge the different approaches of Wikipedia and OSM -> both have their pros and cons. As a contributor i am more comfortable> with the OSM approach but i would never say that collecting information> from secondary sources is inherently less valuable than collecting> empiric data.>>>  I wonder whether OSM is really always based on original research, or>>  whether we sometimes just want to believe this.>> Well - human perception is of course always connected to our past> experience and beliefs, we cannot observe our environment in a truly> neutral way. And of course with armchair mapping much of the data in> OSM is produced by people who have never been near the place they map> based on the belief that what the image layer they use depicts what is> actually there (which is sometimes not the case - like with the iconic> demolished buildings) and that what people think they see in the images> actually resembles the impression they would have on the ground (which> is also frequently not the case - prominent example would be here:> http://www.openstreetmap.org/relation/7142214). But the key is every> information in OSM has to be able to stand up to scrutiny by local> mappers - a local mapper standing in front of the object needs to be> able to tell if it is correct or not - even if different mappers could> in borderline cases come to different assessments on the matter.>> --> Christoph Hormann> http://www.imagico.de/>> ___> talk mailing list> talk@openstreetmap.org> https://lists.openstreetmap.org/listinfo/talk___talk mailing listtalk@openstreetmap.orghttps://lists.openstreetmap.org/listinfo/talk___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Sun, Oct 1, 2017 at 3:45 PM, Tomas Straupis 
wrote:

> > Tomas, you claimed that "It adds NO value."  This is demonstrably wrong.
> You
> > are right that the same fixing was done for years. But until wikidata
> tag,
> > there was no easy way to FIND them.
>
>   There always was.
>   You simply take wikipedia provided geo-tags dump like
> https://dumps.wikimedia.org/ltwiki/latest/ltwiki-latest-geo_tags.sql.gz
>
>   This gives you a very simple table with: lat/lon/page_title.
>   No parsing or anything else involved.
>   You then take data from OSM - lat/lon/wikipedia.tag
>   So you have two tables of same structure. Voila. You can compare
> anything (title, coordinates), in any direction with some
> approximation if needed etc. No OSM wikidata involved at all.
>

Thomas, this will not work. Matching wikidata & osm by coordinates is
useless, because the coordinates differ too much -- see the hard data proof
in the prev email.  The only way you can make any useful calculation is if
you analyze the entirety of Wikidata graph, and merge it with OSM objects,
and expose it to other users so that they can figure out what is right or
broken.  That's exactly what my Wikidata+OSM service allows users to do.

>   If wikipedia page moves - title is gone from this dump and the new
> one appears on the same coordinates. You can map them very quickly.
> Theoretically you can update OSM data automatically, but usually if
> wikipedia title has changed, it means that something has changed in
> the object on the ground, so maybe something else has to be changed in
> OSM data as well (for example name).
>

Again - not possible - because coordinate matching is mostly useless.
Also, no, usually wikipedia titles change not because something changed on
the ground, but because of a conflict with a similarly named place
somewhere else. People usually rename the original page to a more specific
name, and create a new page in its place listing all the disambiguations.
This is what breaks titles most often. We now have about 800 left (after
thousands already fixed), plus potentially thousands more of those that
have not been tagged with wikidata tag yet.

>
> I'm just saying the same could be done without wikidata tags.
>

As explained by me in one of the first emails, and by Andy, and a few
others, it cannot be done **as easily**. You can build a complex system if
you have enough disk space (~1TB), and do a local resolve of wikipedia ->
wikidata, and build a complex service on top of it.  Or you can simply add
a single tag that has already been added to 90% of cases, and use
off-the-shelve query engine to merge the data, and let everyone use it.

>
>   See above. What are practical advantages of your method?
>   Because theoretically you are taking a set A, creating a new set B
> from this A, and then you're trying to fix A according to B. This is
> logical nonsense :-) There is no point of putting this B into OSM.
> This is a temporary data which could be stored in your local "error
> checking" database.
>

Strawman argument :)   For each object that has a tag, I use JOSM to get
corresponding wikidata tag, and upload that data to OSM.  The moment it is
uploaded, other systems, such as my wikidata+osm service, get that data.
Then community, without my involvement, can analyze the data with many
different queries, and fix all the errors they find.  If I haven't uploaded
the data to OSM, only I would be able to see it, and only I would be able
to fix it.  I don't know all the different ways community may query the
data (I'm already getting hundreds of thousands of queries). Its a tool
that helps community.

>
>   550 objects globally... Well... :-) You should see from here, that
> the problem is finding people who want to FIX, not finding problems...
>

750 is number NOW. It used to be many thousands. And was all fixed, by
volunteers. For just the most obvious of queries.  There are many more
fixes that needs to happen - see wikipedia link cleanup project on osm
wiki.  So once the problems are identified, they get solved. Finding them
is the problem.

> I'm arguing against idea that wikipedia tag is outdated or in any way
> worse.

But this is exactly what I have been showing with my data about broken
tags. Do you have any data to say that it is not worse?

> Yes, OSM would not be born
> without a geek idea, but it would not have reached what it is now if
> it would not be easy to understand for non geeks. Wikidata tag is
> totally non-understandable to non-geeks.
>

Wikidata does not need to be understood by geeks or non-geeks. It's an ID,
and everyone understands that concept, and most people don't touch tags
they don't understand. Just like mapillary ID, or tons of other local
government IDs.  The tools we have, like iD editor, can easily work with
these IDs without non-geeks as you call them understanding it. The query
system also doesn't need to be understood to be used - you simply share the
link

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Paul Norman


On 10/1/2017 5:39 PM, Yuri Astrakhan wrote:
Lastly, if the coordinates are different, you may not copy it from OSM 
to Wikidata because of the difference in the license.


Just for clarity and anyone reading the archives later, copying from 
Wikidata to OSM is also a problem because Wikidata permits coordinate 
sources like Wikipedia or Google Earth.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

John, I guess it is always good to talk as a data scientist - with numbers
and facts. Here's why matching by coordinates would not work.  This query
calculates the distance between the OSM nodes, and the coordinates that
Wikidata has for those nodes. I only looked at nodes, because ways and
relations are even more incorrect - Wiki only has a center point.  The
results are bucketed by the distance (in km) - the bigger the distance, the
bigger the mismatch between OSM and Wikipedia.   As you can see,  only a
small number of nodes are accurate to 10 meters.Query:
http://tinyurl.com/ybp4tp7a

diff in km number of nodes
<0.01 75,027
<0.1 131,644
<0.5 147,637
<1 46,891
<2 28,049
<5 10,792
<10 3,537
10+ 7,239

Is this a convincing argument why we should have a Wikipedia/Wikidata link,
as oppose to calculate it?

The other issue is why we need Wikidata links - while I have said it many
times, let me say it again.  Because the current system is badly broken -
as is evident by tens of thousands of errors that my approach has
uncovered.  I am not advocating to delete Wikipedia tag. Only that when you
use wikipedia tag, it creates a burden on the community to maintain, and
community is clearly unable to keep up with the changes on the Wikipedia
side. So instead of using just the bad link (page title), I am advocating
to use a good link (wikidata).  We are already using it for 90%. Why not
fill in the last 10%?  It does not change anything of how you do your
mapping. It simply helps those who want to fix errors, or view
corresponding wikipedia articles even if it gets renamed.

On Sun, Oct 1, 2017 at 8:50 PM, john whelan  wrote:

> >Assuming my above arguments has convinced you
>
> No I still do not see a requirement here, but there again I'm only part of
> the community and that's the concern you appear to be ramming this down our
> threats.  As for what iD does or does not do, I don't see that is relevant.
>
> Why does OSM need it and why are you unable to put forth a convincing
> argument that is accepted by the community?   A ninety percent acceptance
> rate will be fine but I'm not seeing it.
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread john whelan

>Assuming my above arguments has convinced you

No I still do not see a requirement here, but there again I'm only part of
the community and that's the concern you appear to be ramming this down our
threats.  As for what iD does or does not do, I don't see that is relevant.

Why does OSM need it and why are you unable to put forth a convincing
argument that is accepted by the community?   A ninety percent acceptance
rate will be fine but I'm not seeing it.

Cheerio John

On 1 October 2017 at 20:39, Yuri Astrakhan  wrote:

> On Sun, Oct 1, 2017 at 8:15 PM, john whelan  wrote:
>
>> Since an OSM object has  lat and long value and it appears that wiki
>> whatever also has one the entries can be linked.
>>
>
> Not so.  The data is very often different between wikipedia, wikidata, and
> OSM. Also, the same location could be a square, a famous sculpture within
> that square, and some commemorative plaque on it, and all could have some
> wikipedia/wikidata entry. Matching them up requires humans, and cannot
> reliably be done by an algorithm in a large number of cases. Lastly, if the
> coordinates are different, you may not copy it from OSM to Wikidata because
> of the difference in the license.
>
>>
>> "This gives you a very simple table with: lat/lon/page_title.
>>   No parsing or anything else involved.
>>   You then take data from OSM - lat/lon/wikipedia.tag
>>   So you have two tables of same structure. Voila. You can compare
>> anything (title, coordinates), in any direction with some
>> approximation if needed etc. No OSM wikidata involved at all."
>>
>
> See above,  this cannot be done with any reasonable reliability by
> automatic means. You will end up with an incredible amount of unreliable
> data. Feel free to discuss deleting of both Wikipedia and Wikidata tags,
> but I seriously doubt the community will go for it.
>
>>
>> I really don't understand why wikidata needs to be added.  Note the word
>> need, I'm missing the requirement somehow that overides following normal
>> OSM practices.
>>
>
> Assuming my above arguments has convinced you -- that we must manually
> determine the match between an OSM feature and a Wikipedia article, lets
> discuss how best to link to Wikipedia.  There are two options: link by
> article title, and link by Wikidata ID. The first one causes many errors -
> because titles get renamed, and old titles are reused for other meanings.
> The second approach is less readable when looking at the tag, but it is
> much more stable.  Its as simple as that.  One approach causes errors, the
> other approach is more stable.  Both point to Wikipedia article, just using
> a slightly different URL internally.
>
> Automatically adding Wikidata tags is already being done by iD. I would
> like to finish that process, so that the community can clean up all the
> mistakes that are hiding in the OSM db.
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Sun, Oct 1, 2017 at 8:15 PM, john whelan  wrote:

> Since an OSM object has  lat and long value and it appears that wiki
> whatever also has one the entries can be linked.
>

Not so.  The data is very often different between wikipedia, wikidata, and
OSM. Also, the same location could be a square, a famous sculpture within
that square, and some commemorative plaque on it, and all could have some
wikipedia/wikidata entry. Matching them up requires humans, and cannot
reliably be done by an algorithm in a large number of cases. Lastly, if the
coordinates are different, you may not copy it from OSM to Wikidata because
of the difference in the license.

>
> "This gives you a very simple table with: lat/lon/page_title.
>   No parsing or anything else involved.
>   You then take data from OSM - lat/lon/wikipedia.tag
>   So you have two tables of same structure. Voila. You can compare
> anything (title, coordinates), in any direction with some
> approximation if needed etc. No OSM wikidata involved at all."
>

See above,  this cannot be done with any reasonable reliability by
automatic means. You will end up with an incredible amount of unreliable
data. Feel free to discuss deleting of both Wikipedia and Wikidata tags,
but I seriously doubt the community will go for it.

>
> I really don't understand why wikidata needs to be added.  Note the word
> need, I'm missing the requirement somehow that overides following normal
> OSM practices.
>

Assuming my above arguments has convinced you -- that we must manually
determine the match between an OSM feature and a Wikipedia article, lets
discuss how best to link to Wikipedia.  There are two options: link by
article title, and link by Wikidata ID. The first one causes many errors -
because titles get renamed, and old titles are reused for other meanings.
The second approach is less readable when looking at the tag, but it is
much more stable.  Its as simple as that.  One approach causes errors, the
other approach is more stable.  Both point to Wikipedia article, just using
a slightly different URL internally.

Automatically adding Wikidata tags is already being done by iD. I would
like to finish that process, so that the community can clean up all the
mistakes that are hiding in the OSM db.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread john whelan

Rather than fill OSM up with automated edits that have not even been
discussed with the local community can we think more about functionality?

Since an OSM object has  lat and long value and it appears that wiki
whatever also has one the entries can be linked.

"This gives you a very simple table with: lat/lon/page_title.
  No parsing or anything else involved.
  You then take data from OSM - lat/lon/wikipedia.tag
  So you have two tables of same structure. Voila. You can compare
anything (title, coordinates), in any direction with some
approximation if needed etc. No OSM wikidata involved at all."

I really don't understand why wikidata needs to be added.  Note the word
need, I'm missing the requirement somehow that overides following normal
OSM practices.

Cheerio John

On 1 Oct 2017 7:19 pm,  wrote:

> Hi everybody.
>
> We already accepted wikipedia links keep in mind that wiki article isn't
> the same abstraction as OSM object.
> And the way we make a reference on wiki articles differs over time.
> It was a link, it was an article name, it was a name with language prefix.
>
> Wikidata id is a way to make a refererence onto wikipedia article
> which wiki community states to be the right and consistent one.
> Why don't we just accept the recomended way of referencing wiki articles?
>
> That doesn't make wikipedia articles more or less verifable on the ground.
> But that makes the link persistent agains the changes in wikipedia.
>
> 01.10.2017, 16:38, "Christoph Hormann" :
> > On Sunday 01 October 2017, Marc Gemis wrote:
> >>  If I noticed an inception date on a information sign next to a
> >>  building, is this original research or a secondary source ?
> >
> > The date on the sign is verifiable as a signed date, not necessarily as
> > a date connected to the building. Historic information like dates from
> > before the timeframe witnessed by today's mappers is something that is
> > problematic in OSM in general.
> >
> > As any historian will be able to tell you history as a science is not
> > about analyzing events of the past in the same way you analyze present
> > day observations in empiric science disciplines but about analyzing and
> > interpreting sources - in this way it resembles the approach of
> > Wikipedia more than that of OSM.
> >
> >>  If a Wikidata contributor/Wikipedian researches websites like Orbis,
> >>  Onroerend erfgoed, etc and finds different inception dates and
> >>  records them all, is it bad that s/he uses secondary sources instead
> >>  of the information sign on the ground ?
> >
> > I don't want to judge the different approaches of Wikipedia and OSM -
> > both have their pros and cons. As a contributor i am more comfortable
> > with the OSM approach but i would never say that collecting information
> > from secondary sources is inherently less valuable than collecting
> > empiric data.
> >
> >>  I wonder whether OSM is really always based on original research, or
> >>  whether we sometimes just want to believe this.
> >
> > Well - human perception is of course always connected to our past
> > experience and beliefs, we cannot observe our environment in a truly
> > neutral way. And of course with armchair mapping much of the data in
> > OSM is produced by people who have never been near the place they map
> > based on the belief that what the image layer they use depicts what is
> > actually there (which is sometimes not the case - like with the iconic
> > demolished buildings) and that what people think they see in the images
> > actually resembles the impression they would have on the ground (which
> > is also frequently not the case - prominent example would be here:
> > http://www.openstreetmap.org/relation/7142214). But the key is every
> > information in OSM has to be able to stand up to scrutiny by local
> > mappers - a local mapper standing in front of the object needs to be
> > able to tell if it is correct or not - even if different mappers could
> > in borderline cases come to different assessments on the matter.
> >
> > --
> > Christoph Hormann
> > http://www.imagico.de/
> >
> > ___
> > talk mailing list
> > talk@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread dkiselev

Hi everybody.

We already accepted wikipedia links keep in mind that wiki article isn't the 
same abstraction as OSM object.
And the way we make a reference on wiki articles differs over time.
It was a link, it was an article name, it was a name with language prefix.

Wikidata id is a way to make a refererence onto wikipedia article 
which wiki community states to be the right and consistent one.
Why don't we just accept the recomended way of referencing wiki articles?

That doesn't make wikipedia articles more or less verifable on the ground.
But that makes the link persistent agains the changes in wikipedia.

01.10.2017, 16:38, "Christoph Hormann" :
> On Sunday 01 October 2017, Marc Gemis wrote:
>>  If I noticed an inception date on a information sign next to a
>>  building, is this original research or a secondary source ?
>
> The date on the sign is verifiable as a signed date, not necessarily as
> a date connected to the building. Historic information like dates from
> before the timeframe witnessed by today's mappers is something that is
> problematic in OSM in general.
>
> As any historian will be able to tell you history as a science is not
> about analyzing events of the past in the same way you analyze present
> day observations in empiric science disciplines but about analyzing and
> interpreting sources - in this way it resembles the approach of
> Wikipedia more than that of OSM.
>
>>  If a Wikidata contributor/Wikipedian researches websites like Orbis,
>>  Onroerend erfgoed, etc and finds different inception dates and
>>  records them all, is it bad that s/he uses secondary sources instead
>>  of the information sign on the ground ?
>
> I don't want to judge the different approaches of Wikipedia and OSM -
> both have their pros and cons. As a contributor i am more comfortable
> with the OSM approach but i would never say that collecting information
> from secondary sources is inherently less valuable than collecting
> empiric data.
>
>>  I wonder whether OSM is really always based on original research, or
>>  whether we sometimes just want to believe this.
>
> Well - human perception is of course always connected to our past
> experience and beliefs, we cannot observe our environment in a truly
> neutral way. And of course with armchair mapping much of the data in
> OSM is produced by people who have never been near the place they map
> based on the belief that what the image layer they use depicts what is
> actually there (which is sometimes not the case - like with the iconic
> demolished buildings) and that what people think they see in the images
> actually resembles the impression they would have on the ground (which
> is also frequently not the case - prominent example would be here:
> http://www.openstreetmap.org/relation/7142214). But the key is every
> information in OSM has to be able to stand up to scrutiny by local
> mappers - a local mapper standing in front of the object needs to be
> able to tell if it is correct or not - even if different mappers could
> in borderline cases come to different assessments on the matter.
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On 1 October 2017 at 18:29, Tomas Straupis  wrote:

>> Also, please elaborate which community has asked me to stay away???
>
>   Lithuania.

Please can you point to the place where this was discussed and
consensus reached, also to where that was communicated to the wider
community?

> We are in active action on not only fixing wikipedia
> tags, but also adding missing tags to OSM, adding missing coordinates
> to wikipedia, aligning coordinates between OSM and wikipedia etc. For
> YEARS!

Me too.

That does not preclude the work Yuri is doing; the two are not
mutually exclusive.

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Stefano

2017-10-01 21:45 GMT+02:00 Tomas Straupis :

>
>
> >>   When we create a POI detail page, we want to add a link (url without
> >> redirects) to a wikipedia page. To do that it is straightforward to
> >> use a value in wikipedia tag.
> > Great, thanks. As you can see, nothing in what I do breaks that.
> Instead, it
> > actually helps your POI links to be more accurate.
>
>   It does not help. We are not using wikidata in any way. We are
> fixing wikipedia links, OSM objects, wikipedia articles manually using
> automated checks described above to pinpoint the problems.
>
> *You* aren't using Wikidata.
If you want a redirect to a wikipedia page to build an url, here it is
https://www.wikidata.org/wiki/Special:GoToLinkedPage?site=enwiki=Q936

Instead of filling the database with hundreds of variations of wikipedia
tag (wikipedia, wikipedia:, wikipedia=:* and so on) you need
one to build the one you need.



> --
> Tomas
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

>>   It is mostly because you pushed the effort, not beaucse of
>> "advantage of wikidata". The same fixing has already been done for
>> YEARS before your effors based on wikipedia tags only.
>
>
> Tomas, you claimed that "It adds NO value."  This is demonstrably wrong. You
> are right that the same fixing was done for years. But until wikidata tag,
> there was no easy way to FIND them.

  There always was.
  You simply take wikipedia provided geo-tags dump like
https://dumps.wikimedia.org/ltwiki/latest/ltwiki-latest-geo_tags.sql.gz

  This gives you a very simple table with: lat/lon/page_title.
  No parsing or anything else involved.
  You then take data from OSM - lat/lon/wikipedia.tag
  So you have two tables of same structure. Voila. You can compare
anything (title, coordinates), in any direction with some
approximation if needed etc. No OSM wikidata involved at all.

  If wikipedia page moves - title is gone from this dump and the new
one appears on the same coordinates. You can map them very quickly.
Theoretically you can update OSM data automatically, but usually if
wikipedia title has changed, it means that something has changed in
the object on the ground, so maybe something else has to be changed in
OSM data as well (for example name).

> About a year ago when I first started
> this project, I created lists of thousands of such errors, that were very
> rapidly fixed once they were identified. This was not possible before.

  Cool. I have nothing against that. I'm just saying the same could be
done without wikidata tags.

> My method is for finding broken wikipedia tags. What method are you talking
> about? Can you describe what method you use to identify errors?

  See above.

> Here is the DATA for my "theoretical ramblings". Can you show any data to
> back your theoretical ramblings?

  See above. What are practical advantages of your method?
  Because theoretically you are taking a set A, creating a new set B
from this A, and then you're trying to fix A according to B. This is
logical nonsense :-) There is no point of putting this B into OSM.
This is a temporary data which could be stored in your local "error
checking" database.

> Now there is a simpler http://tinyurl.com/ybv7q7n6 query - it used to have
> about 1300, now down to ~750. And these are JUST the disambig errors.

  550 objects globally... Well... :-) You should see from here, that
the problem is finding people who want to FIX, not finding problems...

> wiki. Lastly, what am I proposing to destroy?!?  I am ADDING a tag and
> ADDING a new search mechanism, because there is current no reliable
> mechanism to fix these things.

  I have nothing against that. I'm arguing against idea that wikipedia
tag is outdated or in any way worse. Yes, OSM would not be born
without a geek idea, but it would not have reached what it is now if
it would not be easy to understand for non geeks. Wikidata tag is
totally non-understandable to non-geeks.

> This is wonderful that you are fixing all these issues, could you tell me
> how you find them? Also, funny enough, I used to live in Vilnius a long time
> ago, near Gineitiškės. Should I be allowed to edit there? (I hope this
> doesn't lead to another huge but unrelated discussion :) )

  You know that is not the point. You could still live in Lithuania
and you would still need to consult the local community before doing
automated changes.

>>   When we create a POI detail page, we want to add a link (url without
>> redirects) to a wikipedia page. To do that it is straightforward to
>> use a value in wikipedia tag.
> Great, thanks. As you can see, nothing in what I do breaks that. Instead, it
> actually helps your POI links to be more accurate.

  It does not help. We are not using wikidata in any way. We are
fixing wikipedia links, OSM objects, wikipedia articles manually using
automated checks described above to pinpoint the problems.

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Sun, Oct 1, 2017 at 1:29 PM, Tomas Straupis 
wrote:

> 2017-10-01 20:04 GMT+03:00 Yuri Astrakhan:
> >> 2. Its not a WORK to automatically update one osm tag according to
> another
> >> osm tag (anybody can do it online/locally/etc). It adds NO value.
> >
> > It adds HUGE value, as was repeatably shown. Thanks to Wikidata IDs, the
> > community was able to see and fix tens of thousands of errors in
> Wikipedia
> > tags. <...>
>
>   It is mostly because you pushed the effort, not beaucse of
> "advantage of wikidata". The same fixing has already been done for
> YEARS before your effors based on wikipedia tags only.

Tomas, you claimed that "It adds NO value."  This is demonstrably wrong.
You are right that the same fixing was done for years. But until wikidata
tag, there was no easy way to FIND them.  About a year ago when I first
started this project, I created lists of thousands of such errors, that
were very rapidly fixed once they were identified. This was not possible
before.

> And fixing
> wikipedia tags is in no way inferior to your method. Maybe even
> better, because it involves less „geekiness“ - they are more
> understandable to larger portion of OSM community.
>
> My method is for finding broken wikipedia tags. What method are you
talking about? Can you describe what method you use to identify errors?

>> 3. It is totally unacceptable to introduce idea that wikipedia tag could
> >> be removed at some time, because some other new automatically filled
> tag has
> >> been introduced.
> >
> > First, it is always acceptable to introduce and discuss new ideas. Any
> > ideas. Always. <...>
>
>   Yes. But when you're told by numerous people numerous times that
> current mechanism works, and there is nothing BETTER in your advice
> (other than your theoretical rambilngs), you cannot advice to destroy
> existing working mechanism.
>

Here is the DATA for my "theoretical ramblings". Can you show any data to
back your theoretical ramblings?
  https://commons.wikimedia.org/w/index.php?title=Data:
Sandbox/Yurik/OSM_objects_pointing_to_disambigs.tab=history
Now there is a simpler http://tinyurl.com/ybv7q7n6 query - it used to have
about 1300, now down to ~750. And these are JUST the disambig errors. There
are many other types as I listed in the Wikipedia improvement project on
osm wiki. Lastly, what am I proposing to destroy?!?  I am ADDING a tag and
ADDING a new search mechanism, because there is current no reliable
mechanism to fix these things.

> > We are discussing the way to improve them,
> > because they are currently broken. Badly.
>
>   And they are perfectly being fixed without involving wikidata tags
> there, where people WANT to do that and do WORK to fix them.
>
> Do you have any data to back that up?  When I first looked at them,
Wikipedia links were often incorrect (see links above). Now they are fixed
thanks to all the work done by the communities. Yes, all that manual work
that people did. But in order to WORK, you need to FIND issues first.

> > Also, please elaborate which community has asked me to stay away???
>
>   Lithuania. We are in active action on not only fixing wikipedia
> tags, but also adding missing tags to OSM, adding missing coordinates
> to wikipedia, aligning coordinates between OSM and wikipedia etc. For
> YEARS!
>

This is wonderful that you are fixing all these issues, could you tell me
how you find them? Also, funny enough, I used to live in Vilnius a long
time ago, near Gineitiškės. Should I be allowed to edit there? (I hope this
doesn't lead to another huge but unrelated discussion :) )

>
>   When we create a POI detail page, we want to add a link (url without
> redirects) to a wikipedia page. To do that it is straightforward to
> use a value in wikipedia tag.
>

Great, thanks. As you can see, nothing in what I do breaks that. Instead,
it actually helps your POI links to be more accurate.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 20:04 GMT+03:00 Yuri Astrakhan:
>> 2. Its not a WORK to automatically update one osm tag according to another
>> osm tag (anybody can do it online/locally/etc). It adds NO value.
>
> It adds HUGE value, as was repeatably shown. Thanks to Wikidata IDs, the
> community was able to see and fix tens of thousands of errors in Wikipedia
> tags. <...>

  It is mostly because you pushed the effort, not beaucse of
"advantage of wikidata". The same fixing has already been done for
YEARS before your effors based on wikipedia tags only. And fixing
wikipedia tags is in no way inferior to your method. Maybe even
better, because it involves less „geekiness“ - they are more
understandable to larger portion of OSM community.

>> 3. It is totally unacceptable to introduce idea that wikipedia tag could
>> be removed at some time, because some other new automatically filled tag has
>> been introduced.
>
> First, it is always acceptable to introduce and discuss new ideas. Any
> ideas. Always. <...>

  Yes. But when you're told by numerous people numerous times that
current mechanism works, and there is nothing BETTER in your advice
(other than your theoretical rambilngs), you cannot advice to destroy
existing working mechanism.

> We are discussing the way to improve them,
> because they are currently broken. Badly.

  And they are perfectly being fixed without involving wikidata tags
there, where people WANT to do that and do WORK to fix them.

> Also, please elaborate which community has asked me to stay away???

  Lithuania. We are in active action on not only fixing wikipedia
tags, but also adding missing tags to OSM, adding missing coordinates
to wikipedia, aligning coordinates between OSM and wikipedia etc. For
YEARS!

> 4) could you elaborate on who uses wikipedia
> tags, and how they are being used? It would greatly help to understand
> various use cases for such data.

  When we create a POI detail page, we want to add a link (url without
redirects) to a wikipedia page. To do that it is straightforward to
use a value in wikipedia tag.

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Sun, Oct 1, 2017 at 11:12 AM, Tomas Straupis 
wrote:

> I guess the point is that:
> 1. Its ok to play with some pet-tag like wikidata
>
100 % agree

> 2. Its not a WORK to automatically update one osm tag according to another
> osm tag (anybody can do it online/locally/etc). It adds NO value.
>

It adds HUGE value, as was repeatably shown. Thanks to Wikidata IDs, the
community was able to see and fix tens of thousands of errors in Wikipedia
tags. The Wikipedia link improvement project is based on it:
https://wiki.openstreetmap.org/wiki/Wikipedia_Link_Improvement_Project
Also, there is no point to add it online/locally because that wouldn't help
community to find these errors.

3. It is totally unacceptable to introduce idea that wikipedia tag could be
> removed at some time, because some other new automatically filled tag has
> been introduced.
>

First, it is always acceptable to introduce and discuss new ideas. Any
ideas. Always. We, as a community, don't have to accept them, but
discussing innovations is always a good thing.  That said, the removal of
wikipedia tag is NOT being discussed here. We are discussing the way to
improve them, because they are currently broken. Badly.

> So if you like wikidata tag - go ahead and enjoy it, but do not tuch
> wikipedia tag with autoscripts because people are actually using it.
> Especially when you not only avoid discussing with local communities, but
> ignore active requests from local communities to stay away.
>
> Tomas, 1) i don't have autoscripts to touch wikipedia tag, I use JOSM to
generate wikidata tags because of the benefits it provides 2) i am
providing a way for community to fix the broken wikipedia tags, 3) I have
been very actively talking to many communities (in, ru, de, fr, ...). Also,
please elaborate which community has asked me to stay away??? A very broad
statement, considering that every single community had many members
supporting this effort. And 4) could you elaborate on who uses wikipedia
tags, and how they are being used? It would greatly help to understand
various use cases for such data.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

I guess the point is that:
1. Its ok to play with some pet-tag like wikidata
2. Its not a WORK to automatically update one osm tag according to another
osm tag (anybody can do it online/locally/etc). It adds NO value.
3. It is totally unacceptable to introduce idea that wikipedia tag could be
removed at some time, because some other new automatically filled tag has
been introduced.

So if you like wikidata tag - go ahead and enjoy it, but do not tuch
wikipedia tag with autoscripts because people are actually using it.
Especially when you not only avoid discussing with local communities, but
ignore active requests from local communities to stay away.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On 1 October 2017 at 14:03, Andy Townsend  wrote:
> On 01/10/2017 13:05, Andy Mabbett wrote:

>> And now you're making things up.
>>
> just two posts earlier in this thread you said
>
>> I don't think this kind of sarcastic hyperbole helps to move things
>> forward, nor to engender an atmosphere of collegial collaboration.
>> Please resist the temptation to use it.
>
> I would respectfully suggest that you follow your own advice.

I do; my comment was not "sarcastic hyperbole", but a factual observation.

> There's a valid discussion to be had about "how OSM does things vs how
> wikipedia/wikidata does things"

Has anyone said there is not?

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Andy Townsend

On 01/10/2017 13:05, Andy Mabbett wrote:

And now you're making things up.

just two posts earlier in this thread you said

> I don't think this kind of sarcastic hyperbole helps to move things
> forward, nor to engender an atmosphere of collegial collaboration.
> Please resist the temptation to use it.

I would respectfully suggest that you follow your own advice.

There's a valid discussion to be had about "how OSM does things vs how 
wikipedia/wikidata does things".  Back in 2016 in another context I 
mentioned "Common End" in Derbyshire on this list 
https://lists.openstreetmap.org/pipermail/talk/2016-November/077139.html 
.  It's in wikipedia as "a place noted on a map" (which is correct - OS 
maps include it).  It doesn't in any verifiable sense "exist" though.  
Wikidata has it https://www.wikidata.org/wiki/Q5153341 as both a 
"hamlet" and a "fictional location".  Whether that's "correct" or not is 
a decision for wikidata - I've no idea what their definition of "hamlet" 
is and whether it includes a locality that probably used to exist in 
some sense but all on-the-ground trace of the name has disappeared, but 
it's entirely reasonable to discuss the areas in which different 
contribution customs will result in different data, and how we handle 
links in those cases.

Best Regards,

Andy

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On 1 October 2017 at 12:13, Christoph Hormann  wrote:

> the consensus in the OSM community to link to Wikipedia articles, and
> other useful third-party sources?

> I find it fascinating how both you and Yuri seem to be eager to always
> deflect the discussion from the main subject, namely the automated
> editing/addition of wikidata IDs

Your 337-word post to which I replied, mentioned Wikidata only once,
tangentially, mentioned Wikipedia nine times, and was mostly concerned
with your thesis on your perception of "the fundamental differences
between OSM and Wikipedia."

> and misinterpreting constructive
> critique of that as an attempt to tell local mappers what tags they may
> and may not add to the things they map.

And now you're making things up.

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk
-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Christoph Hormann

On Sunday 01 October 2017, Marc Gemis wrote:
>
> If I noticed an inception date on a information sign next to a
> building, is this original research or a secondary source ?

The date on the sign is verifiable as a signed date, not necessarily as 
a date connected to the building.  Historic information like dates from 
before the timeframe witnessed by today's mappers is something that is 
problematic in OSM in general.

As any historian will be able to tell you history as a science is not 
about analyzing events of the past in the same way you analyze present 
day observations in empiric science disciplines but about analyzing and 
interpreting sources - in this way it resembles the approach of 
Wikipedia more than that of OSM.

> If a Wikidata contributor/Wikipedian researches websites like Orbis,
> Onroerend erfgoed, etc and finds different inception dates and
> records them all, is it bad that s/he uses secondary sources instead
> of the information sign on the ground ?

I don't want to judge the different approaches of Wikipedia and OSM - 
both have their pros and cons.  As a contributor i am more comfortable 
with the OSM approach but i would never say that collecting information 
from secondary sources is inherently less valuable than collecting 
empiric data.

> I wonder whether OSM is really always based on original research, or
> whether we sometimes just want to believe this.

Well - human perception is of course always connected to our past 
experience and beliefs, we cannot observe our environment in a truly 
neutral way.  And of course with armchair mapping much of the data in 
OSM is produced by people who have never been near the place they map 
based on the belief that what the image layer they use depicts what is 
actually there (which is sometimes not the case - like with the iconic 
demolished buildings) and that what people think they see in the images 
actually resembles the impression they would have on the ground (which 
is also frequently not the case - prominent example would be here: 
http://www.openstreetmap.org/relation/7142214).  But the key is every 
information in OSM has to be able to stand up to scrutiny by local 
mappers - a local mapper standing in front of the object needs to be 
able to tell if it is correct or not - even if different mappers could 
in borderline cases come to different assessments on the matter.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Christoph Hormann

On Sunday 01 October 2017, Andy Mabbett wrote:
>
> Or perhaps it is you who is "deep denial" about the consensus in the
> OSM community to link to Wikipedia articles, and other useful
> third-party sources?

May i suggest you to read my previous messages on this thread to find 
the answer to that question?

I find it fascinating how both you and Yuri seem to be eager to always 
deflect the discussion from the main subject, namely the automated 
editing/addition of wikidata IDs and misinterpreting constructive 
critique of that as an attempt to tell local mappers what tags they may 
and may not add to the things they map.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On 1 October 2017 at 10:06, Christoph Hormann  wrote:
> On Sunday 01 October 2017, Yuri Astrakhan wrote:

> Wow, i had to read this twice to really believe what i was reading here.

I don't think this kind of sarcastic hyperbole helps to move things
forward, nor to engender an atmosphere of collegial collaboration.
Please resist the temptation to use it.

> Seems you are still in deep denial about the fundamental differences
> between OSM and Wikipedia.

Or perhaps it is you who is "deep denial" about the consensus in the
OSM community to link to Wikipedia articles, and other useful
third-party sources?

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Marc Gemis

> Wikipedia is based on secondary sources, it rejects original research.
> OSM is fundamentally different in that because it is based on
> verification by original research.

I'm trying to understand this.

If I noticed an inception date on a information sign next to a
building, is this original research or a secondary source ?

If a Wikidata contributor/Wikipedian researches websites like Orbis,
Onroerend erfgoed, etc and finds different inception dates and records
them all, is it bad that s/he uses secondary sources instead of the
information sign on the ground ?

I wonder whether OSM is really always based on original research, or
whether we sometimes just want to believe this.

regards

m

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

Christoph, I am not talking about OSM or Wikidata or Wikipedia quality or
approaches. Please don't read more into it than what I am trying to state.

If we say that we want OSM objects to link to Wikipedia (and we clearly do,
judging by the number of wikipedia tags people have created), we need a
good way to do it.

Linking to Wikipedia with the page titles is bad. It is not stable.
Wikidata tags fixes that.  No other claim is being made here.

On Sun, Oct 1, 2017 at 5:06 AM, Christoph Hormann  wrote:

> On Sunday 01 October 2017, Yuri Astrakhan wrote:
> >
> > Wikipedia created a stable ID system for these pages. Its called
> > Wikidata. Please view Wikidata as first and foremost a linking system
> > to Wikipedia articles.  [...]
> >
> > Andy, you keep saying Wikidata is not verifiable data - but that's
> > because you keep insisting on separating it from Wikipedia.
>
> Wow, i had to read this twice to really believe what i was reading here.
> Seems you are still in deep denial about the fundamental differences
> between OSM and Wikipedia.
>
> Wikipedia is based on secondary sources, it rejects original research.
> Therefore you can find a lot of nonsense on Wikipedia - all kind of
> urban legends and things like that, especially about remote areas, as
> long as everyone believes them and no one bothers to proof them wrong
> and rebut them outside of Wikipedia.  So in a way Wikipedia documents
> societies current beliefs about the world, not the world itself.  This
> does not necessarily have to go as far as an article about something
> fictitious claiming to be about a real world thing, often its smaller
> stuff like X being an object of type Y.  The iconic 'citation needed'
> of Wikipedia is not about the information being in need of actual
> verification as a fact, it is about this information being verified to
> be something well integrated into societies' belief system.
>
> OSM is fundamentally different in that because it is based on
> verification by original research.  This does not mean everything in
> OSM holds up to this standard but we aim for this and value information
> that is practically verifiable by local mappers and tagging concepts
> that are targeted at verifiable mapping more than other information
> that people always will keep adding to OSM to some extent despite it
> being non-verifiable.
>
> It also means information in OSM is inherently more variable because
> what people observe on the ground varies - both because what people see
> depends on their experience and background and because appearance of
> reality, especially of natural features, varies over time.  OSM with
> its original research research focus lacks the unifying and consistency
> preserving effect of the filter through secondary sources you have in
> Wikipedia.
>
> What you do when you mechanically 'fix errors' and correct discrepancies
> between tags in OSM that contradict the Wikipedia/Wikidata information
> is you impose the value system of Wikipedia onto OSM.
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-10-01 Thread Christoph Hormann

On Sunday 01 October 2017, Yuri Astrakhan wrote:
>
> Wikipedia created a stable ID system for these pages. Its called
> Wikidata. Please view Wikidata as first and foremost a linking system
> to Wikipedia articles.  [...]
>
> Andy, you keep saying Wikidata is not verifiable data - but that's
> because you keep insisting on separating it from Wikipedia.

Wow, i had to read this twice to really believe what i was reading here.  
Seems you are still in deep denial about the fundamental differences 
between OSM and Wikipedia.

Wikipedia is based on secondary sources, it rejects original research.  
Therefore you can find a lot of nonsense on Wikipedia - all kind of 
urban legends and things like that, especially about remote areas, as 
long as everyone believes them and no one bothers to proof them wrong 
and rebut them outside of Wikipedia.  So in a way Wikipedia documents 
societies current beliefs about the world, not the world itself.  This 
does not necessarily have to go as far as an article about something 
fictitious claiming to be about a real world thing, often its smaller 
stuff like X being an object of type Y.  The iconic 'citation needed' 
of Wikipedia is not about the information being in need of actual 
verification as a fact, it is about this information being verified to 
be something well integrated into societies' belief system.

OSM is fundamentally different in that because it is based on 
verification by original research.  This does not mean everything in 
OSM holds up to this standard but we aim for this and value information 
that is practically verifiable by local mappers and tagging concepts 
that are targeted at verifiable mapping more than other information 
that people always will keep adding to OSM to some extent despite it 
being non-verifiable.

It also means information in OSM is inherently more variable because 
what people observe on the ground varies - both because what people see 
depends on their experience and background and because appearance of 
reality, especially of natural features, varies over time.  OSM with 
its original research research focus lacks the unifying and consistency 
preserving effect of the filter through secondary sources you have in 
Wikipedia.

What you do when you mechanically 'fix errors' and correct discrepancies 
between tags in OSM that contradict the Wikipedia/Wikidata information 
is you impose the value system of Wikipedia onto OSM.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-30 Thread Yuri Astrakhan

Verifiability is critical to OSM success, but it does not mean it must only
be  verifiable by visiting the physical location. Tags like "wikipedia",
"wikidata", "url", "website" and some IDs cannot be verified that way.  You
must visit some external website to validate. Stopping by Yellowstone
National Park or a statue in the middle of a city may tell you its national
registration number, but most likely you will have to visit some government
website. Seeing some complex URL tells you nothing about its correctness
unless you visit that web site.

Yet, we are not talking about the last two examples.  Node 153699914 has
wikipedia="Eureka, Wisconsin".  It looks fine to a casual examiner, but in
reality is a garbage link to a disambiguation place - a list of 3 different
places, which you wouldn't know unless you visit the external site -
Wikipedia.  I have uncovered many thousands of such cases, and many of them
have already been fixed thanks to a stronger IDing system.  Yet, every day
there is more of them - because Wikipedia keeps renaming things, and
several people refuse to allow Wikidata IDs.

Wikipedia created a stable ID system for these pages. Its called Wikidata.
Please view Wikidata as first and foremost a linking system to Wikipedia
articles.  It is NOT perfect. It has many issues. But it is simply much
better than linking to Wikipedia articles by their names because they don't
break as often.

Andy, you keep saying Wikidata is not verifiable data - but that's because
you keep insisting on separating it from Wikipedia. We can already make it
so that when you click on Wikidata link, you are taken directly to
Wikipedia. The statements on Wikidata entries are a major bonus for
automated verification and other things, but it should be viewed in
addition to the redirecting capability, not as a replacement to Wikipedia
pages.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-29 Thread Andy Townsend


On 28/09/2017 07:15, Stefano wrote:



We used this library to process the dump and then we add the results 
in pgsql


https://www.entropywins.wtf/blog/2015/11/08/wikidata-wikibase-json-dump-reader/

https://github.com/osmItalia/wikidata-geo-match

What would the requirement of a wikidata2pgsql be?


Thanks for that.

To answer the question, It'd create a database in a format that's 
designed to be queried that contains "just enough" information to 
support whatever job it's needed for (and it'd be great if it also 
supported dynamic column creation using a mechanism similar to 
osm2pgsql's ".style" file).


Of the other "missing bits", I used "osmosis" as an example of "cutting 
a database extract down to size" (other options are available).  In the 
OSM world that initial slice is often geographical, but osmosis can also 
deal with data without explicit co-ordinates (ways and relations) based 
on the geographical location of constituent nodes.  The same would be 
true (for me) of wikidata - I'd be only interested in actual physical 
locations and the things that they link to (which may not have physical 
locations and may just be concepts).


Finally "switch2osm" is a regularly-updated set of instructions that you 
can follow from start to finish without needed external knowledge about 
how to solve a problem.  For example, 
https://switch2osm.org/manually-building-a-tile-server-16-04-2-lts/ has 
undergone numerous updates in the last year to deal with stylesheet 
changes, which depend on a bleeding edge version of carto, which depends 
on  node.js.  At each stage in the process the idea was that 
you'd always be able to get a working result, even if at one point that 
meant the instructions explained how get a version of the stylesheet 
from a few months ago because a newer version wouldn't work in 
combination with everything else there.


"wikidata-wikibase-json-dump-reader" looks interesting - it looks (to 
continue the analogy) to be somewhat equivalent to the Crosby PBF library.


"wikidata-geo-match" also looks interesting because 
https://github.com/osmItalia/wikidata-geo-match/blob/master/scripts/2_process.sh 
and the readme explain how to do the initial geographical selection.  
It's not quite all there though 
(https://github.com/osmItalia/wikidata-geo-match/search?utf8=%E2%9C%93=0_create_wikidata_table.sql= 
suggests either the README is out of date or some bits are missing).  
It'd certainly be a useful start for someone who cared about wikidata to 
develop something mirroring the equivalent tools that OSM already has, 
and I'm sure it does exactly what you need it to do, but it's not a 
generic "let's create a database and allow you to do something with it".


Best Regards,

Andy


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-28 Thread Stefano

On 28 Sep 2017 2:57 am, "Andy Townsend"  wrote:


It depends - if you want to do a "quick search for something" then an
equivalent to overpass turbo might be an option, but in the real world what
you'd _actually_ want to do is a local database query.  Unfortunately that
side of things seems to be completely missing (or at least very
well-hidden) - wikidata seems to be quite immature in that respect.
Where's the "switch2osm" for wikidata?  Where's the "osm2pgsql" or
"osmosis"?  Sure I can download 20Gb of gzipped JSON from
https://dumps.wikimedia.org/wikidatawiki/entities/20170925/ and try and
write some sort of parser based on https://www.mediawiki.org/
wiki/Wikibase/DataModel/JSON , but this seems very much like going back to
banging the rocks together (and no, a third-party query interface that
depends on an external network connection such as
https://query.wikidata.org/ or anything else isn't a better option).


We used this library to process the dump and then we add the results in
pgsql

https://www.entropywins.wtf/blog/2015/11/08/wikidata-wikibase-json-dump-reader/

https://github.com/osmItalia/wikidata-geo-match

What would the requirement of a wikidata2pgsql be?


Regards,
Andy


Ciao,
Stefano



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-27 Thread Andy Townsend


On 26/09/2017 18:08, Yuri Astrakhan wrote:



  When data consumers want to get a link to corresponding wikipedia
article, doing that with wikipedia[:xx] tags is straightforward. Doing
the same with wikidata requires additional pointless and time
consuming abrakadabra.


no, you clearly haven't worked with any data consumers recently. Data 
consumers want Wikidata, much more than wikipedia tags - please talk 
to them.


That would be me in a former job, I think.

One of the things that I used to spend a lot of time doing was finding 
ways to encode data so that knowledge could be shared by e.g. field 
engineers, and then analysing those results so that you can find out 
what was related to what, what caused what, and how much store you can 
set by a particular result or prediction.  There are a couple of points 
worth sharing from that experience:


1) The first point to make about human-contributed data is that it's 
variable.  Some people will say something is probably an X, some people 
probably a Y.  The reality is that they're actually both right some of 
the time.  You might think (in the context of e.g. shop brands) "hang on 
- surely a shop can be only one brand?  It must be _either_ X or Y!" but 
you'd be wrong.  There are _always_ exceptions, and there will always be 
"errors" - you just don't know which way is right and which wrong.


2) The second point that's relevant here is that codes such as CODE1, 
CODE2 etc. are to be avoided at all costs since they don't enable any 
natural visualisation of what's been captured.  You have already said 
"but surely every system that displays data can look up the description" 
but anyone familar with the real world knows that that simply won't 
happen.  This means that there's no way for an ordinary mapper to verify 
whether the magic code on an OSM item is correct or not.  Verifiability 
is one of the key concepts of OSM (see 
https://wiki.openstreetmap.org/wiki/Verifiability et al) and anything 
that moves away from it means that data isn't going to be maintained, 
because people simply won't understand what it means.  I suspect that a 
key part of the success of OSM was the reliance on natural 
language-based keys and values, and a loose tagging scheme that allowed 
easy expansion.


3) The third point is that a database that has been "cleaned" so that 
there are no "errors" in it is worth far less than one that hasn't, when 
you're trying to understand the complex relationships between objects.  
This goes against most normal data processing instincts because 
obviously normally you'd try and ensure that data has full referential 
integrity - but where there are edge cases (and as per (1) above there 
are always edge cases) different consumers will very likely want to 
treat those edge cases differently, which they can't do if someone has 
"helpfully" merged all the edge cases into more popular categories.



To be blunt, if I was trying to process OSM data and had a need to get 
into the wikidata/wikipedia world based on it (for example because I 
wanted the municipal coat of arms - something not in OSM) I'd take a 
wikipedia link over a wikidata one every time because all mappers will 
have been able to see the text of the wikipedia link rather than just 
something like Q123456.  You've made the point that things change in 
wikipedia regularly (articles get renamed etc.), but it's important to 
remember that things change in the real world all the time as well - and 
a link that's suddenly pointing at something different in wikipedia is 
immediately apparent, in the same way that if Q123456 was no longer 
relevant (because the real world thing has changed) it wouldn't be.


All that said, I don't see wikidata as a key component (or even a very 
useful component) of OSM - but we all map things that are of interest to 
us - some people map in great detail the style of British telephone 
boxes or the "Royal Cipher" on postboxes which I see absolutely no point 
in, but if it's verifiable, why not - I'm sure I'm mapping stuff that is 
irrelevent to them.  A problem with wikidata (as noted above) is that 
I'm not sure that it _is_ verifiable data - I suspect it'll get stale 
after adding and never be maintained, simply because people will never 
notice that it's wrong.


(and on an unrelated comment in the same message)



Sure, it can be via dump parsing, but it is a much more complicated 
than querying.  Would you rather use Overpass turbo to do a quick 
search for some weird thing that you noticed, or download and parse 
the dump?  Most people would rather do the former.


It depends - if you want to do a "quick search for something" then an 
equivalent to overpass turbo might be an option, but in the real world 
what you'd _actually_ want to do is a local database query. 
Unfortunately that side of things seems to be completely missing (or at 
least very well-hidden) - wikidata seems to be quite immature in that 
respect.   Where's

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-27 Thread Yuri Astrakhan

I have been fixing nodes that have wikipedia but no wikidata tags [1], and
even the first two randomly picked nodes had identical problem - article
was renamed (twice!) without leaving redirects  - node 1136510320

Try it yourself - run the query and see what the it points to.
[1]
https://wiki.openstreetmap.org/wiki/Wikipedia_Link_Improvement_Project#Missing_Wikidata_tags

Imre, I think at this point it might be better to have both, just as a
safety check. But I can already see that they get misaligned - articles
keep getting renamed, so we will be stuck mindlessly updating wikipedia
tag. Feels a bit like a busywork for the sake of work, but might be needed
for a bit.

On Wed, Sep 27, 2017 at 6:00 PM, Imre Samu  wrote:

> > I hope everyone realizes that there are Wikidata items for which there
> > is no Wikipedia article.
> > So you cannot always find it via Wikipedia  tags.
> >  And at least JOSM shows a human readable name of a Wikidata item
> > besides the Q-number. I think iD does this as well.
> > m. (who manually adds Wikidata references for Flemish churches after
> creating the Wikidata items).
>
> imho:
> probably you have a local and domain knowledge on the topic of "Flemish
> churches"
> but for me:  wikidata without wikipedia page - is  extremely suspicious
>
> because:
>
> #1.  Sometimes the " nearby" search for geolocated articles/wikidataids is
> not enough
> for example:
> * at least ~28000 churches exist in the wikidata without coordinates:
> http://tinyurl.com/y8nyk9zw
>
> And probably we will also find wikidata cities without coordinates.
>
> #2. And we should aware of the current "Parallel geo worlds" problem in
> the wikidata[1]
> for example:
> Arad ( major City in Romania ) has 3 wikidata, and we should prefer id
> with wikipedia pages.
> * https://www.wikidata.org/wiki/Q173591 ( with wikipedia pages, linked to
> OSM )
> * https://www.wikidata.org/wiki/Q31886684 (  created by Cebuano import
> [1] ~1 month ago) * https://www.wikidata.org/wiki/Q16898082
>
> [1] wikidata cebuano import problem:
> * https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/
> 2017/08#Dealing_with_our_second_planet * https://www.wikidata.org/wiki/
> Wikidata:Project_chat/Archive/2017/08#Nonsense_imported_from_Geonames
>
>
> Imre
>
>
>
>
>
>
> 2017-09-27 5:03 GMT+02:00 Marc Gemis :
>
>> On Wed, Sep 27, 2017 at 1:17 AM, Andy Townsend  wrote:
>> > That's simply rubbish.  Tags on an OSM object describe it in the real
>> world.
>> > They should be verifiable.  Whether an OSM object has a wikidata tag on
>> it
>> > is essentially irrelevant as far as OSM is concerned - it's just a
>> primary
>> > key into an external database.  External data consumers might find the
>> data
>> > in that database useful, but they can also get to it via wikipedia tags
>> > (which, being human-readable, are more likely to be maintained), so it's
>> > really not a big deal.
>>
>>
>> I hope everyone realizes that there are Wikidata items for which there
>> is no Wikipedia article. So you cannot always find it via Wikipedia
>> tags.
>>
>> And at least JOSM shows a human readable name of a Wikidata item
>> besides the Q-number. I think iD does this as well.
>>
>> m. (who manually adds Wikidata references for Flemish churches after
>> creating the Wikidata items).
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-27 Thread Imre Samu

> I hope everyone realizes that there are Wikidata items for which there
> is no Wikipedia article.
> So you cannot always find it via Wikipedia  tags.
>  And at least JOSM shows a human readable name of a Wikidata item
> besides the Q-number. I think iD does this as well.
> m. (who manually adds Wikidata references for Flemish churches after
creating the Wikidata items).

imho:
probably you have a local and domain knowledge on the topic of "Flemish
churches"
but for me:  wikidata without wikipedia page - is  extremely suspicious

because:

#1.  Sometimes the " nearby" search for geolocated articles/wikidataids is
not enough
for example:
* at least ~28000 churches exist in the wikidata without coordinates:
http://tinyurl.com/y8nyk9zw

And probably we will also find wikidata cities without coordinates.

#2. And we should aware of the current "Parallel geo worlds" problem in the
wikidata[1]
for example:
Arad ( major City in Romania ) has 3 wikidata, and we should prefer id with
wikipedia pages.
* https://www.wikidata.org/wiki/Q173591 ( with wikipedia pages, linked to
OSM )
* https://www.wikidata.org/wiki/Q31886684 (  created by Cebuano import [1]
   ~1 month ago) * https://www.wikidata.org/wiki/Q16898082

[1] wikidata cebuano import problem:
* https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/
2017/08#Dealing_with_our_second_planet * https://www.wikidata.org/wiki/
Wikidata:Project_chat/Archive/2017/08#Nonsense_imported_from_Geonames


Imre






2017-09-27 5:03 GMT+02:00 Marc Gemis :

> On Wed, Sep 27, 2017 at 1:17 AM, Andy Townsend  wrote:
> > That's simply rubbish.  Tags on an OSM object describe it in the real
> world.
> > They should be verifiable.  Whether an OSM object has a wikidata tag on
> it
> > is essentially irrelevant as far as OSM is concerned - it's just a
> primary
> > key into an external database.  External data consumers might find the
> data
> > in that database useful, but they can also get to it via wikipedia tags
> > (which, being human-readable, are more likely to be maintained), so it's
> > really not a big deal.
>
>
> I hope everyone realizes that there are Wikidata items for which there
> is no Wikipedia article. So you cannot always find it via Wikipedia
> tags.
>
> And at least JOSM shows a human readable name of a Wikidata item
> besides the Q-number. I think iD does this as well.
>
> m. (who manually adds Wikidata references for Flemish churches after
> creating the Wikidata items).
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-27 Thread Marc Gemis

So you do not agree with the Automated Edits code of conduct ?
If an automated edit takes place in a country, why do you expect that
that community follows the talk mailing list or even speak English ?
People has the right to know that some stranger starts making changes
in their area without being forced to read a mailing list (which is an
outdated medium for the younger) or understand English.

An import has to be discussed with the local community, so why would
an automated edit be different ?

regards

m.

On Wed, Sep 27, 2017 at 5:38 PM, Martin Koppenhoefer
 wrote:
>
>
> 2017-09-27 16:56 GMT+02:00 Andy Mabbett :
>>
>> On 26 September 2017 at 21:39, Martin Koppenhoefer
>>  wrote:
>>
>> >> This might also mean that
>> >> you have to discuss it via Telegram, Facebook, email, IRC, etc.
>> >> depending on where that local community is.
>> >>
>> >> The talk mailing list is not sufficient.
>>
>> > I think this is problematic. If the local community uses a paid service
>> > for
>> > communication you'd have to pay in order to speak to your fellow
>> > mappers?
>>
>> The mapping community in the West Midlands of England communicates
>> most often face-to-face, meeting in a a pub.
>>
>> Perhaps we could mandate that anyone wanting to make an automated edit
>> in the region has to buy the beer?
>
>
>
> you don't only pay with money, e.g. in Facebook you only pay money if you
> are a client of theirs (buying visibility for your advertizing), the
> ordinary users (merchandise) pay with data and their privacy.
>
> Cheers,
> Martin
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-27 Thread Martin Koppenhoefer

2017-09-27 16:56 GMT+02:00 Andy Mabbett :

> On 26 September 2017 at 21:39, Martin Koppenhoefer
>  wrote:
>
> >> This might also mean that
> >> you have to discuss it via Telegram, Facebook, email, IRC, etc.
> >> depending on where that local community is.
> >>
> >> The talk mailing list is not sufficient.
>
> > I think this is problematic. If the local community uses a paid service
> for
> > communication you'd have to pay in order to speak to your fellow mappers?
>
> The mapping community in the West Midlands of England communicates
> most often face-to-face, meeting in a a pub.
>
> Perhaps we could mandate that anyone wanting to make an automated edit
> in the region has to buy the beer?
>


you don't only pay with money, e.g. in Facebook you only pay money if you
are a client of theirs (buying visibility for your advertizing), the
ordinary users (merchandise) pay with data and their privacy.

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-27 Thread Andy Mabbett

On 26 September 2017 at 21:39, Martin Koppenhoefer
 wrote:

>> This might also mean that
>> you have to discuss it via Telegram, Facebook, email, IRC, etc.
>> depending on where that local community is.
>>
>> The talk mailing list is not sufficient.

> I think this is problematic. If the local community uses a paid service for
> communication you'd have to pay in order to speak to your fellow mappers?

The mapping community in the West Midlands of England communicates
most often face-to-face, meeting in a a pub.

Perhaps we could mandate that anyone wanting to make an automated edit
in the region has to buy the beer?

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Marc Gemis

On Wed, Sep 27, 2017 at 1:17 AM, Andy Townsend  wrote:
> That's simply rubbish.  Tags on an OSM object describe it in the real world.
> They should be verifiable.  Whether an OSM object has a wikidata tag on it
> is essentially irrelevant as far as OSM is concerned - it's just a primary
> key into an external database.  External data consumers might find the data
> in that database useful, but they can also get to it via wikipedia tags
> (which, being human-readable, are more likely to be maintained), so it's
> really not a big deal.

I hope everyone realizes that there are Wikidata items for which there
is no Wikipedia article. So you cannot always find it via Wikipedia
tags.

And at least JOSM shows a human readable name of a Wikidata item
besides the Q-number. I think iD does this as well.

m. (who manually adds Wikidata references for Flemish churches after
creating the Wikidata items).

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Andy Townsend

On 26/09/2017 03:40, Yuri Astrakhan wrote:

... I have been blocked by Andy Townsend with the following message.

Let's begin at the beginning - this was a "0-hour block" - you weren't
prevented from using the API for _any_ period of time, merely forced to
read this message first. This was a last resort - many other attempts
at communication have been made over at least the last 10 months (since
November 2016 - https://www.openstreetmap.org/changeset/43749373 ). The
issues that I raised back then are still true today - see
https://lists.openstreetmap.org/pipermail/talk/2017-September/078780.html
for more details. It makes no sense to mechanically copy a wikidata
value to OSM when the wikidata object expresses only part of the sense
of the wikipedia page.

Simple example in case things are still not clear:

1) Imagine there are two objects in OSM - a village and an admin area
containing that village.

2) Wikipedia only has one page for a "village and an admin area"
3) The wikidata page (probably created by a bot) is only for the village
4) Linking the OSM admin area to the wikidata page for the village is an
error.

This is the sort of thing that you've been doing again and again for the
last 10 months.

A few interesting semi-relevant statistics so far: the number of
discovered links to disambig pages is now back to over 800, even
without 100k+ untaged ways. And there are almost 38,000 osm objects
where wikipedia tag does not correspond with wikidata tag. The number
is very high, but fixing them should be semi-automated, as most of
them are redirects. TBD.

There are a lots of possibilities here. Maybe the OSM object shouldn't
have a wikipedia entry at all. Maybe it's significantly changed since
the link was added, and should be changed. It needs someone with
real-world knowledge of the OSM object to update the links - anything
else is just guessing, and has no place in OSM.

If by "semi-automated" you mean a human-centric approach like Kort,
MapRoulette, StreetComplete et al then fine - but that's not been your
approach so far.

Here's Andy's message, with my inlined replies. I think that almost
all of the raised points have been raised and answered in our previous
discussion, but I feel it is my responsibility to present them again.

You're conducting an import of known bad data (your own changeset
comments say "Further cleanup will be done using...").

Per previous description, the existing data is already bad, and I am
simply making it possible to identify it, after discussing it on this
thread.
No, that is untrue. See e.g.
https://www.openstreetmap.org/changeset/52002597 .

You are wilfully ignoring the feedback that you're receiving now
and have received in the past. A lot of issues have been raised
about the quality of your edits - see
http://resultmaps.neis-one.org/osm-discussion-comments?uid=339581
. In many cases you seem to agree that you're adding rubbish, and
yet you continue.

You seem to be suggesting (in
https://lists.openstreetmap.org/pipermail/talk/2017-September/078767.html
) that "the community" clean up your mess. This is not the way
that OpenStreetMap works - if an individual is adding data to it
(especially large quantities of data) then it is their
responsibility to ensure that the data that they are adding is
valid, or at least as valid as the data that is already there.

Again, no, I am identifying rubbish, not introducing it, and I am very
actively replying to every comment I receive.
You are not actually _resolving_ any of the problems that people are
finding with the edits that you are making. See for example
https://www.openstreetmap.org/changeset/52341792 . In that example
someone says that you added a wikidata tag in error. You agree that you
added it in error (and in fact a whole category of the tags that you've
added is in error - I've commented on a couple more within the last
hour). You have not done anything to resolve this error that you have
introduced into OSM .

Going further back, in your replies to changeset comments you've said
things like "I have already stopped changing any objects except the
admin levels regions 1-6" https://openstreetmap.org/changeset/4377
but have carried on regardless. Mappers have repeatedly asked you to
use geographically smaller changesets
https://openstreetmap.org/changeset/44078387**https://openstreetmap.org/changeset/44090685
https://openstreetmap.org/changeset/44203236 and yet you continue
regardless.

Either you're incompetent in the changes you're making or you're lying
to us; in neither case should you be continuing to edit as you have been
doing.

... The way to solve the quality of this data is to analyze it with
the OSM+Wikidata tool I have built,
... or with something else that doesn't require OSM to be mechanically
edited by you first. As has already been said

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Martin Koppenhoefer

2017-09-26 5:14 GMT+02:00 Marc Gemis :

> This might also mean that
> you have to discuss it via Telegram, Facebook, email, IRC, etc.
> depending on where that local community is.
>
> The talk mailing list is not sufficient.



I think this is problematic. If the local community uses a paid service for
communication you'd have to pay in order to speak to your fellow mappers?

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Stefano

2017-09-26 19:43 GMT+02:00 Yves :

> I think that the underlying issue in wikidata tags is that they are
> external IDs. Not human readable, they cannot be entered 'by hand' nor
> verified on the ground.
>

Yes, it's an external ID, but it acts as intermediary with other databases,
because Wikidata acts as a central repository for IDs (see
https://cacm.acm.org/magazines/2014/10/178785-wikidata/fulltext). One could
perform a reconciliation of its database against Wikidata instead of
against OSM, and get a match 'for free'.
We can't add OSM ids in WD because they aren't stable, so Wikidata IDs
could be the stable IDs for certain classes of objects (those "worth" of
some description in another project?).

"Entered by hand" they can't be in the same way as Wikipedia entries can't
(you add a wikipedia tag hoping someone will fill the "red link" on that
wikipedia page?)



> Once you accept them in OSM, you can't really complain about bots.
>
> Yves (who still think such UIDs are only needed for the lack of good query
> tools).
>
>
>
Stefano
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Yuri Astrakhan

Yves, yes, they are external IDs. But so are wikipedia titles.  Visually
inspecting Wikipedia tile does not provide you with any way to verify its
correctness - you have to look in the external data source (WP).  As for
entering by hand - just like you shouldn't enter Wikipedia articles by hand
- you should copy/paste it from the article, or use the autocomplete field
in iD.  So in reality, these two things are nearly the same.  On the other
hand, modern rely on the internet connection, which means that an ID can be
shown as text in the user's language, together with other metadata from
Wikidata.  The concept of "internal" vs "external" is not as relevant now
as it was in the past...  (there is only one data - the internet :))

On Tue, Sep 26, 2017 at 1:43 PM, Yves  wrote:

> I think that the underlying issue in wikidata tags is that they are
> external IDs. Not human readable, they cannot be entered 'by hand' nor
> verified on the ground.
> Once you accept them in OSM, you can't really complain about bots.
>
> Yves (who still think such UIDs are only needed for the lack of good query
> tools).
>
>
>
> Le 26 septembre 2017 19:08:33 GMT+02:00, Yuri Astrakhan <
> yuriastrak...@gmail.com> a écrit :
>>
>> > p.s. OSM is a community project, not a programmers project, it's about
>>> > people, not software :-)
>>>
>>> It's both.  OSM is first and foremost is a community, but the result of
>> our effort is a machine-readable database.  We are not creating an
>> encyclopedia that will be casually flipped through by humans. We produce
>> data that gets interpreted by software, so that it can render maps and be
>> searchable.  For example, if every person uses their own tag names and ways
>> to record things, the data will have nearly zero value.  We must agree on
>> conventions so that software can understand our results - which is exactly
>> what we have been doing on wiki and in email channels. Any tag and value
>> that cannot be recognized and processed by software is effectively ignored.
>>
>>
>>>   Totally agree. If some script can automatically add new tag
>>> (wikidata) without any actual WORK needed, then it is pointless,
>>> anybody can run an auto-update script.
>>
>>   When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia
>>> data, they add wikipedia link, not wikidata "stuff".
>>>
>>
>> While sand castles may look nice, they don't last very long. When
>> ordinary people add just the Wikipedia article, that link quickly gets
>> stale and become irrelevant and often incorrect. The wikipedia article
>> titles are not stable. They get renamed all the time - there are tens of
>> thousands of them in OSM already that I found.  Often, community renames wp
>> articles because there are more than one meaning, so they create a new
>> article with the same name in its place - a disambig page.  There is no
>> easy way to analyse wikipedia links for content - you cannot easily
>> determine if the wikipedia article is about a person, a country, or a
>> house, which makes it impossible to check for correctness.
>>
>> When I spend half an hour of my time researching which WP article is best
>> for an object, I do not want that effort to be wasted just because someone
>> else puts a disambig page in its place, and I have to redo all my work.
>>
>>   When data consumers want to get a link to corresponding wikipedia
>>> article, doing that with wikipedia[:xx] tags is straightforward. Doing
>>> the same with wikidata requires additional pointless and time
>>> consuming abrakadabra.
>>>
>>
>> no, you clearly haven't worked with any data consumers recently. Data
>> consumers want Wikidata, much more than wikipedia tags - please talk to
>> them. Wikidata gives you the list of wikipedia articles in all available
>> languages, it lets you get multi-lingual names when they are not specified
>> in OSM, it allows much more intelligent searches based on types of objects,
>> it allows quality controls.  The abrakadabra is exactly what one has to do
>> when parsing non-standardized data.
>>
>>>
>>>   Validation of wikipedia tag values can and IS already done using osm
>>> data versus wikipedia-geolocated data extracts/dumps.
>>>
>>> Sure, it can be via dump parsing, but it is a much more complicated than
>> querying.  Would you rather use Overpass turbo to do a quick search for
>> some weird thing that you noticed, or download and parse the dump?  Most
>> people would rather do the former. Here is the same thing - you *could* do
>> validation via a dump, but that barrier of entry is so high, most people
>> wouldn't.  With the new OSM+Wikidata tool, which is already getting
>> hundreds of thousands requests (!!!) , it is possible to get just the data
>> you need, and fix the problems that have been always present, but hidden.
>> And all that is possible because of a single tag.
>>
>
___
talk mailing list
talk@openstreetmap.org

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Yves

I think that the underlying issue in wikidata tags is that they are external 
IDs. Not human readable, they cannot be entered 'by hand' nor verified on the 
ground. 
Once you accept them in OSM, you can't really complain about bots. 

Yves (who still think such UIDs are only needed for the lack of good query 
tools).


Le 26 septembre 2017 19:08:33 GMT+02:00, Yuri Astrakhan 
 a écrit :
>>
>> > p.s. OSM is a community project, not a programmers project, it's
>about
>> > people, not software :-)
>>
>> It's both.  OSM is first and foremost is a community, but the result
>of
>our effort is a machine-readable database.  We are not creating an
>encyclopedia that will be casually flipped through by humans. We
>produce
>data that gets interpreted by software, so that it can render maps and
>be
>searchable.  For example, if every person uses their own tag names and
>ways
>to record things, the data will have nearly zero value.  We must agree
>on
>conventions so that software can understand our results - which is
>exactly
>what we have been doing on wiki and in email channels. Any tag and
>value
>that cannot be recognized and processed by software is effectively
>ignored.
>
>
>>   Totally agree. If some script can automatically add new tag
>> (wikidata) without any actual WORK needed, then it is pointless,
>> anybody can run an auto-update script.
>
>  When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia
>> data, they add wikipedia link, not wikidata "stuff".
>>
>
>While sand castles may look nice, they don't last very long. When
>ordinary
>people add just the Wikipedia article, that link quickly gets stale and
>become irrelevant and often incorrect. The wikipedia article titles are
>not
>stable. They get renamed all the time - there are tens of thousands of
>them
>in OSM already that I found.  Often, community renames wp articles
>because
>there are more than one meaning, so they create a new article with the
>same
>name in its place - a disambig page.  There is no easy way to analyse
>wikipedia links for content - you cannot easily determine if the
>wikipedia
>article is about a person, a country, or a house, which makes it
>impossible
>to check for correctness.
>
>When I spend half an hour of my time researching which WP article is
>best
>for an object, I do not want that effort to be wasted just because
>someone
>else puts a disambig page in its place, and I have to redo all my work.
>
>  When data consumers want to get a link to corresponding wikipedia
>> article, doing that with wikipedia[:xx] tags is straightforward.
>Doing
>> the same with wikidata requires additional pointless and time
>> consuming abrakadabra.
>>
>
>no, you clearly haven't worked with any data consumers recently. Data
>consumers want Wikidata, much more than wikipedia tags - please talk to
>them. Wikidata gives you the list of wikipedia articles in all
>available
>languages, it lets you get multi-lingual names when they are not
>specified
>in OSM, it allows much more intelligent searches based on types of
>objects,
>it allows quality controls.  The abrakadabra is exactly what one has to
>do
>when parsing non-standardized data.
>
>>
>>   Validation of wikipedia tag values can and IS already done using
>osm
>> data versus wikipedia-geolocated data extracts/dumps.
>>
>> Sure, it can be via dump parsing, but it is a much more complicated
>than
>querying.  Would you rather use Overpass turbo to do a quick search for
>some weird thing that you noticed, or download and parse the dump? 
>Most
>people would rather do the former. Here is the same thing - you *could*
>do
>validation via a dump, but that barrier of entry is so high, most
>people
>wouldn't.  With the new OSM+Wikidata tool, which is already getting
>hundreds of thousands requests (!!!) , it is possible to get just the
>data
>you need, and fix the problems that have been always present, but
>hidden.
>And all that is possible because of a single tag.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Yuri Astrakhan

>
> > p.s. OSM is a community project, not a programmers project, it's about
> > people, not software :-)
>
> It's both.  OSM is first and foremost is a community, but the result of
our effort is a machine-readable database.  We are not creating an
encyclopedia that will be casually flipped through by humans. We produce
data that gets interpreted by software, so that it can render maps and be
searchable.  For example, if every person uses their own tag names and ways
to record things, the data will have nearly zero value.  We must agree on
conventions so that software can understand our results - which is exactly
what we have been doing on wiki and in email channels. Any tag and value
that cannot be recognized and processed by software is effectively ignored.


>   Totally agree. If some script can automatically add new tag
> (wikidata) without any actual WORK needed, then it is pointless,
> anybody can run an auto-update script.

  When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia
> data, they add wikipedia link, not wikidata "stuff".
>

While sand castles may look nice, they don't last very long. When ordinary
people add just the Wikipedia article, that link quickly gets stale and
become irrelevant and often incorrect. The wikipedia article titles are not
stable. They get renamed all the time - there are tens of thousands of them
in OSM already that I found.  Often, community renames wp articles because
there are more than one meaning, so they create a new article with the same
name in its place - a disambig page.  There is no easy way to analyse
wikipedia links for content - you cannot easily determine if the wikipedia
article is about a person, a country, or a house, which makes it impossible
to check for correctness.

When I spend half an hour of my time researching which WP article is best
for an object, I do not want that effort to be wasted just because someone
else puts a disambig page in its place, and I have to redo all my work.

  When data consumers want to get a link to corresponding wikipedia
> article, doing that with wikipedia[:xx] tags is straightforward. Doing
> the same with wikidata requires additional pointless and time
> consuming abrakadabra.
>

no, you clearly haven't worked with any data consumers recently. Data
consumers want Wikidata, much more than wikipedia tags - please talk to
them. Wikidata gives you the list of wikipedia articles in all available
languages, it lets you get multi-lingual names when they are not specified
in OSM, it allows much more intelligent searches based on types of objects,
it allows quality controls.  The abrakadabra is exactly what one has to do
when parsing non-standardized data.

>
>   Validation of wikipedia tag values can and IS already done using osm
> data versus wikipedia-geolocated data extracts/dumps.
>
> Sure, it can be via dump parsing, but it is a much more complicated than
querying.  Would you rather use Overpass turbo to do a quick search for
some weird thing that you noticed, or download and parse the dump?  Most
people would rather do the former. Here is the same thing - you *could* do
validation via a dump, but that barrier of entry is so high, most people
wouldn't.  With the new OSM+Wikidata tool, which is already getting
hundreds of thousands requests (!!!) , it is possible to get just the data
you need, and fix the problems that have been always present, but hidden.
And all that is possible because of a single tag.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Tomas Straupis

> p.s. OSM is a community project, not a programmers project, it's about
> people, not software :-)

  Totally agree. If some script can automatically add new tag
(wikidata) without any actual WORK needed, then it is pointless,
anybody can run an auto-update script.

  When ordinary (non geek) mappers do ACTUAL WORK - add wikipedia
data, they add wikipedia link, not wikidata "stuff".

  When data consumers want to get a link to corresponding wikipedia
article, doing that with wikipedia[:xx] tags is straightforward. Doing
the same with wikidata requires additional pointless and time
consuming abrakadabra.

  Validation of wikipedia tag values can and IS already done using osm
data versus wikipedia-geolocated data extracts/dumps.

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Marc Gemis

Seems, that I was mistaken:

https://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_conduct states:

Either talk (a general purpose mailing list)
or if your edit affects only one country or territory then the
national-language mailing lists, forums, or other standard
communication methods for the territory affected by the change
or ...

This seems odd to me, as not all communities affected by a mechanical
edit are represented in talk. So this means that the country in which
most edits will take place do not have to be consulted ?

regards

m.

p.s. OSM is a community project, not a programmers project, it's about
people, not software :-)

On Tue, Sep 26, 2017 at 10:02 AM, Andy Mabbett
 wrote:
> On 26 September 2017 at 04:14, Marc Gemis  wrote:
>
>> I believe the mechanical edit polity demands that you discuss with the
>> *local* community. That means if your edit modifies items in e.g.
>> Mexico, Belgium and Japan, you have to discuss your edit with the
>> communities in Mexico, Belgium and Japan.
>
> That is clearly impractical for a task of this nature and, if true,
> would be an effective prohibition on any such beneficial editing.
>
> --
> Andy Mabbett
> @pigsonthewing
> http://pigsonthewing.org.uk
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Christoph Hormann

On Tuesday 26 September 2017, Yuri Astrakhan wrote:
> Since this thread had not received any new discussion in the past 4
> days, I assumed all points were answered and proceeded as planned,
> per mechanical edit policy.

That is always a bad idea.  For example in this thread i made the 
following comment:

> > Christoph, a valid point. Yet the duplicate would allow finding
> > many of these errors, rather than leaving wp-only to go bad due to
> > changing nature of the WP articles.
>
> Actually no - you can find the errors just as well without adding the
> wikidata tags to OSM as after doing so.

You did not react to that so why would it make sense for you to assume 
this issue has been resolved - just by ignoring it?

> Per previous description, the existing data is already bad, and I am
> simply making it possible to identify it, after discussing it on this
> thread.

Sorry but this is utter nonsense.  Importing data into OSM is never 
required for either fixing errors in OSM or in the imported data.  If 
you do an import you need to make sure the results are at least on the 
same level of quality as they would be based on competent manual 
mapping.  You need to ensure that in data preparation and not after 
doing the import.

And the presence of pre-existing bad data (to a significant part already 
produced through under-the-radar mechanical edits by the way) is no 
excuse for adding more bad data.

> Andy, Wikidata ID is not correct or incorrect -- [...]

Then it is non-verifiable data and does not belong in the OSM database 
at all.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Andy Mabbett

On 26 September 2017 at 04:14, Marc Gemis  wrote:

> I believe the mechanical edit polity demands that you discuss with the
> *local* community. That means if your edit modifies items in e.g.
> Mexico, Belgium and Japan, you have to discuss your edit with the
> communities in Mexico, Belgium and Japan.

That is clearly impractical for a task of this nature and, if true,
would be an effective prohibition on any such beneficial editing.

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-26 Thread Andy Mabbett

On 26 September 2017 at 03:40, Yuri Astrakhan  wrote:

> I have been blocked by Andy Townsend with the following message.
> I believe Andy is acting in best interest of the project, yet might have
> missed or misread this discussion.

I agree with Yuri.

This series of edits will, once complete, leave OSM in a much better
state than before; with invalid or outdated Wikipedia tags identified
and replaced (or removed); with the bonus of good Wikidata tags added
to them.

Yuri should be unblocked, and allowed to complete the job.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-25 Thread Yuri Astrakhan

Marc, thanks.  I was under the assumption that talk is the global community
- as it is the most generic in the list, unlike talk-us and
talk-us-newyork. Does it meany that any global proposal would require
talking to hundreds of communities independently, making it impossible to
coordinate, because comments in one community would not be visible to other
communities? Is there any kind of ambassadorial program?  Also, does it
mean that talk-us doesn't decide anything because there is a
talk-us-newyork?

In this specific case, adding wikidata seemed like a long overdue task,
something that is already happening automatically by the unmonitored iD
feature.

Btw, I looked at the descriptions at
https://lists.openstreetmap.org/listinfo

On Mon, Sep 25, 2017 at 11:14 PM, Marc Gemis  wrote:

> > moving it here.  I believe I acted in good faith according to the
> mechanical
> > edit policy - discussed with the community, and proceeded.
>
> I believe the mechanical edit polity demands that you discuss with the
> *local* community. That means if your edit modifies items in e.g.
> Mexico, Belgium and Japan, you have to discuss your edit with the
> communities in Mexico, Belgium and Japan. This might also mean that
> you have to discuss it via Telegram, Facebook, email, IRC, etc.
> depending on where that local community is.
>
> The talk mailing list is not sufficient.
>
> regards
>
> m.
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-25 Thread Marc Gemis

> moving it here.  I believe I acted in good faith according to the mechanical
> edit policy - discussed with the community, and proceeded.

I believe the mechanical edit polity demands that you discuss with the
*local* community. That means if your edit modifies items in e.g.
Mexico, Belgium and Japan, you have to discuss your edit with the
communities in Mexico, Belgium and Japan. This might also mean that
you have to discuss it via Telegram, Facebook, email, IRC, etc.
depending on where that local community is.

The talk mailing list is not sufficient.

regards

m.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-25 Thread Yuri Astrakhan

Since this thread had not received any new discussion in the past 4 days, I
assumed all points were answered and proceeded as planned, per mechanical
edit policy. Yet, after I have added all the nodes and moved on to
relations, I have been blocked by Andy Townsend with the following message.
I believe Andy is acting in best interest of the project, yet might have
missed or misread this discussion.  Also, the block is such that I am no
longer able to even reply on the changesets to the raised questions, so
moving it here.  I believe I acted in good faith according to the
mechanical edit policy - discussed with the community, and proceeded.

A few interesting semi-relevant statistics so far:  the number of
discovered links to disambig pages is now back to over 800, even without
100k+ untaged ways. And there are almost 38,000 osm objects where wikipedia
tag does not correspond with wikidata tag. The number is very high, but
fixing them should be semi-automated, as most of them are redirects. TBD.

Here's Andy's message, with my inlined replies. I think that almost all of
the raised points have been raised and answered in our previous discussion,
but I feel it is my responsibility to present them again.

You're conducting an import of known bad data (your own changeset comments
> say "Further cleanup will be done using...").
>

Per previous description, the existing data is already bad, and I am simply
making it possible to identify it, after discussing it on this thread.


> You are wilfully ignoring the feedback that you're receiving now and have
> received in the past. A lot of issues have been raised about the quality of
> your edits - see
> http://resultmaps.neis-one.org/osm-discussion-comments?uid=339581 . In
> many cases you seem to agree that you're adding rubbish, and yet you
> continue.
>
You seem to be suggesting (in
> https://lists.openstreetmap.org/pipermail/talk/2017-September/078767.html
> ) that "the community" clean up your mess. This is not the way that
> OpenStreetMap works - if an individual is adding data to it (especially
> large quantities of data) then it is their responsibility to ensure that
> the data that they are adding is valid, or at least as valid as the data
> that is already there.
>

Again, no, I am identifying rubbish, not introducing it, and I am very
actively replying to every comment I receive.  This is not "my data" - the
data is already in OSM in the form of the incorrect wikipedia tags. This
action is identical to what iD editor does - it *automatically* adds
corresponding wikidata ID, without any additional checks, and without many
users even being aware of it.  The way to solve the quality of this data is
to analyze it with the OSM+Wikidata tool I have built, to see the
mismatches.  Since there are tens (hundreds?) of thousands of issues
already in the database, it is clearly impossible to fix it by one person.
The available choices are:  me doing it by hand, and fixing a handful, or
make it possible to find problems, so everyone can fix them. (per Andy
Mabbett explanation)

Please go back and reread some of your previous replies on
> http://resultmaps.neis-one.org/osm-discussion-comments?uid=339581 .
> Things like "I will mostly work on high level objects (admin level <= 6)"
> suggests that you are at the very least being disingenuous in your dealings
> with the OSM community.
>

This was written a long time ago, before this effort was even started, and
before I have built the tools (OSM+Wikidata) to let community find issues.
Back then I had to do everything myself, and since it was clearly
impossible, I stopped after fixing the wast majority of the uncovered
issues by hand.


> Please stop this mechanical edit now and instead spend your time
> addressing the issues that have been raised.
>

I believe i have answered this numerous times above and in previous
conversations.  I cannot address tens of thousands of issues i *find*, I
can only help community see them, and do my part in fixing them.  Without
this effort, all the bad data in the form of incorrect wikipedia tags will
still be there, quickly rotting away with every wikipedia page rename.

P.S.  An interesting point was brought by Andy in the later online chat:

>
> in the case of https://www.openstreetmap.org/changeset/43749373 the
> errors were explicitly introduced by you.  The links from OSM to wikipedia
> were correct, the thing (probably a bot) creating the wikidata from
> wikipedia didn't understand the breadth of what the wikipedia article
> represented, and you incorrectly linked from OSM to the wikidata article.
>

Andy, Wikidata ID is not correct or incorrect -- it is simply a number
assigned to a Wikipedia article.  That number may have other statements,
which themselves may be incorrect. Adding Wikidata ID locks that Wikipedia
tag in place, to keep it from going stale - in case that page is renamed,
and in case a disambig is created in its place.  In some cases, the concept
presented in

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-22 Thread Eugene Alvin Villar

On Thu, Sep 21, 2017 at 9:22 PM, Martin Koppenhoefer  wrote:

> This is one of the fields (fundamental to OSM), where wikidata is just a
> mess: distinction of geographically localized communities and
> administrative territorial entities.
>
> Just a few examples:
> https://www.wikidata.org/wiki/Q123705 neighbourhood is a subclass of
> "human settlement" and "community". So far so good, but then it is also
> "part of municipality"?
> https://www.wikidata.org/wiki/Q2983893 quarter is a subclass of
> neightbourhood and administrative territorial entity. And it is an instance
> of "designation for an administrative territorial entity".
> https://www.wikidata.org/wiki/Q486972 human settlement looks OK at a
> glance
> https://www.wikidata.org/wiki/Q515 city is a subclass of "human
> settlement", "administrative territorial entity" and "political territorial
> entity" (are these AND or OR?).
> https://www.wikidata.org/wiki/Q3957 town is a subclass of "human
> settlement". It is "part of a country". It was also a subclass of political
> territorial entity until today [1]
> https://www.wikidata.org/wiki/Q532 village is a subclass of "rural
> settlement" and part of "rural area".
> https://www.wikidata.org/wiki/Q14788575 rural settlement is an instance
> of "designation for an administrative territorial entity" and a subclass of
> "human settlement"
> etc.
>
> Take the town example: it has been for some years a subclass of political
> territorial entity and isn't anymore since today. There are tens of
> thousands of objects that are all instances of towns according to wikidata.
> With one edit all of them have lost their "political territorial entity"
> status.
>

I wouldn't worry too much about these very generic classes of human
settlements or administrative areas. It would be better to focus our
attention on the actual Wikidata items on settlements and administrative
areas of each country, like Waldhufendorfs in Germany (
https://www.wikidata.org/wiki/Q351190) or comunes in Italy (
https://www.wikidata.org/wiki/Q747074). These national subclasses should
then be classified as subclasses of whichever appropriate generic
settlements or administrative areas there are.

So I doubt that tens of thousands of towns in Wikidata suddenly lost their
political territorial entity status. I would think that some of them are
still classified as such because of a different path up the ontology tree.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-21 Thread Martin Koppenhoefer

2017-09-20 19:07 GMT+02:00 Christoph Hormann :

> Don't assume such cases are just a freak anomaly - they are not.  OSM
> and wikidata are two very different projects which developed in very
> different contexts.  Just another example: For most cities and larger
> towns (at least in Germany) there exists an admin_level 6/8 unit with
> the same name and most of these seem to have a single wikidata item
> while in OSM we have two separate concepts for the populated place
> (place=city/town) and the administrative unit (boundary relation with
> boundary=administrative + admin_level=6/8).



+1
This is one of the fields (fundamental to OSM), where wikidata is just a
mess: distinction of geographically localized communities and
administrative territorial entities.

Just a few examples:
https://www.wikidata.org/wiki/Q123705 neighbourhood is a subclass of "human
settlement" and "community". So far so good, but then it is also "part of
municipality"?
https://www.wikidata.org/wiki/Q2983893 quarter is a subclass of
neightbourhood and administrative territorial entity. And it is an instance
of "designation for an administrative territorial entity".
https://www.wikidata.org/wiki/Q486972 human settlement looks OK at a glance
https://www.wikidata.org/wiki/Q515 city is a subclass of "human
settlement", "administrative territorial entity" and "political territorial
entity" (are these AND or OR?).
https://www.wikidata.org/wiki/Q3957 town is a subclass of "human
settlement". It is "part of a country". It was also a subclass of political
territorial entity until today [1]
https://www.wikidata.org/wiki/Q532 village is a subclass of "rural
settlement" and part of "rural area".
https://www.wikidata.org/wiki/Q14788575 rural settlement is an instance of
"designation for an administrative territorial entity" and a subclass of
"human settlement"
etc.

Take the town example: it has been for some years a subclass of political
territorial entity and isn't anymore since today. There are tens of
thousands of objects that are all instances of towns according to wikidata.
With one edit all of them have lost their "political territorial entity"
status.


Cheers,
Martin



[1]
https://www.wikidata.org/w/index.php?title=Q3957=revision=563002649=552128156
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Marc Gemis

As for the Faroer Islands (https://www.wikidata.org/wiki/Q4628) I see
some contradicting information there.The labels refer to an
archipelago, while the "is a" statement refers to an administrative
part of Denmark.

When an item has only 1 "is a" statement, it is not possible to refer
to 2 different concepts. However, it is possible to have to "is a"
statements on an item.

m.

On Wed, Sep 20, 2017 at 6:02 PM, Christoph Hormann  wrote:
> On Wednesday 20 September 2017, Eugene Alvin Villar wrote:
>>
>> This is most certainly a wrong modeling in Wikidata. While we can
>> just have one well-written and comprehensive Wikipedia article about
>> the country/archipelago, in Wikidata, one item should correspond to
>> one concept.
>
> Maybe - but even if that is the case the wikidata concept of a country
> or archipelago is not necessarily the same as in OSM.  For countries
> and archipelagos this might sound strange (an archipelago is an
> archipelago, right?) but as you surely know the meaning of tags in OSM
> can be quite peculiar in the way it develops over time based on mapping
> needs and it would be quite insane if wikidata copied all these
> peculiarities in their classification system.
>
> I don't know a lot about wikidata but as far as i can see every
> wikipedia article links to exactly one wikidata item and there are many
> geographic wikipedia articles that describe several different concepts
> together for which separate OSM features exist.
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Simon Poole

Am 20.09.2017 um 20:55 schrieb Yuri Astrakhan:

> While the WMF does not claim any rights in wikidata contents, it
> does not make any representations (one way or the other) as to
> third party rights in the data. As an illustration: you could dump
> all of OSM in to wikidata and the WMF would not need to change or
> do anything. 
>
>
> But the same works in reverse, doesn't it?  Wikidata project, just
> like WP and OSM, is user contributable. If a user uploads data that
> violates project's license, it should be deleted. And for that reason,
> both Wikidata and OSM state the license under which the data is
> contributed and shared. If I make an edit to OSM by copying data from
> Google, wouldn't that be the same thing?
The WMFs doctrine is that data (even more than one item) is not
protectable, the wording on the WD edit page is ambiguous and the ToU
don't really address the issue at all. Further the WMF is not known for
policing wikidata (contrary to the OSMF and OSM)  and it is doubtful if
it could even be done in any reasonable way. Skipping that lots of WD
data was originally derived from WP with its own set of issues.

That said, as long as we don't start using wikidata instead of data from
OSM contributors, it really is just the WMFs  problem. not ours. We
really really have better use for brain power than trying to fix the
WMFs problems for them.

>  
>
>> (CC0), but the reverse depends on if the OSM contributor agreed
>> to dedicate their edits to public domain.
> There is not really a practical and meaningful way in which an OSM
> contributor could do that, outside of facts that they have
> surveyed themselves and kept separate.
>
>
> How I hate to diverge from the main topic, but alas... :)  This does
> sound like a severe problem (that should be taken to a separate
> thread) - if I, as a user, set the Public Domain checkbox, my
> assumptions are that my contributions are PD. If I trace something
> based on some image data, I need to specify that source, otherwise I
> am in violation of the source's license. If I did not specify the
> source, and I checked the PD box, it can be assumed that I am donating
> under PD. If this is not the case, it is a violation of my
> contributor's rights - because otherwise my intention is not being
> honored (i want other people to be able to use my work unrestricted). 
> If anyone wants to comment, please start a new thread :)

This has really been beaten to death: at best the PD flag can be taken
as an indication of sentiment. Fixing it would require re-wording the
actual text, going back to 4 million odd users and asking them to
reconsider their choice. This however would not address the already
mentioned fundamental issues with data prior to such a change (assuming
that it would be practical to implement all the technical measures that
you are suggesting going forward) and further would still run afoul of
the fact that the OSMF doesn't have a mandate, is not even allowed, to
distribute contributed data on any other terms than those compatible
with the contributor terms.

NOT GOING TO HAPPEN (at least not on volunteer time).

Note on the side: If we were to undertake anything even remotely on the
scale of what the above would imply, it is likely that we would review
our current licence instead. However as has been pointed out many times
that would not result is us switching to a non-attribution licence (aka
CC0 or similar), so it wouldn't really help with wikidata compatibility.

>
>> Without it, OSM data is licensed under ODbL, and cannot be
>> copied. We should make it easier to detect what piece of OSM data
>> is in PD.  I do like your USB analogy :) About names - you will
>> be surprised to discover that MB and other places are actively
>> pursuing Wikidata integration because WD tends to have a huge
>> names list, possibly bigger than OSM itself?
>>
>>
> That is nice for MB, but problematic in more than one way for OSM.
>
> Please elaborate, I know of at least one more company that is actively
> doing that.   Sigh, another side topic :D

Very simple: use of wikidata is not declared and not obvious to the end
users, errors in wikidata get attributed to OSM but can't be fixed in
OSM, well can't be fixed without a lot of technical mumbo-jumbo that you
cannot expect non-seasoned hands to know. And even if the user in the
end finds out where to fix an issue, they are spending time fixing
wikidata, not OSM.

It is completely clear that we are in a competitive situation for mind
share, money and contributors (more exact: for contributors time) with
many other players. Now OSM proper has been loosing out big time on the
first point as of late, but luckily hasn't had great requirements on the
2nd (that is why we are still around), but even OSM is not so daft to
want a situation in which it actively has to redirect potential OSM
contributors to a third party to fix "its" core data.

Simon

PS:

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

>
> people fixing WD won’t necessarily check if their fixes work well with
> OSM. Maybe we should include versions in our WD tags?
> I’ve seen OSM objects linked from WD, are there people monitoring changes
> to linked objects?
>
Yes, that's what the Wikidata+OSM service is for. It allows community to
create queries that verify various aspects of OSM objects as related to
Wikidata Objects. For example, if Wikidata object changes its "instance-of"
to disambig, the query would immediatelly flag corresponding OSM object as
having a problematic wiki link.

>
> I think it’s better to add the WD links slowly, verifying on a one by one
> basis that the objects describe the same thing. And having this done for
> some time I can tell that quite often WD items are very basic and defined
> besides their name only by the content of their WP article links, which in
> different languages not always describe the same thing/s.
> If you look into the things there’s a lot to fix in both projects, adding
> WD tags automatically in one go might help less than people doing it
> carefully and fixing the problems on the way.
>
> iD editor has been doing exactly that for substantial time.  Whenever user
adds a wikipedia tag, corresponding Wikidata tag is added automatically. I
seriously doubt there are any (or any at all) people who check that
wikidata ID is correct by hand.  Yet the number of errors that were caught
by cross-linking the data is very significant.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Martin Koppenhoefer

sent from a phone

> On 20. Sep 2017, at 17:37, Eugene Alvin Villar  wrote:
> 
> This is most certainly a wrong modeling in Wikidata. While we can just have 
> one well-written and comprehensive Wikipedia article about the 
> country/archipelago, in Wikidata, one item should correspond to one concept. 
> So there should be separate Wikidata items for the archipelago and the 
> country. The fact that it isn't like that right now is simply because 
> Wikidata is an ongoing project, just like OSM.

people fixing WD won’t necessarily check if their fixes work well with OSM. 
Maybe we should include versions in our WD tags?
I’ve seen OSM objects linked from WD, are there people monitoring changes to 
linked objects?

I think it’s better to add the WD links slowly, verifying on a one by one basis 
that the objects describe the same thing. And having this done for some time I 
can tell that quite often WD items are very basic and defined besides their 
name only by the content of their WP article links, which in different 
languages not always describe the same thing/s.
If you look into the things there’s a lot to fix in both projects, adding WD 
tags automatically in one go might help less than people doing it carefully and 
fixing the problems on the way.

cheers,
Martin 
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

>
> While the WMF does not claim any rights in wikidata contents, it does not
> make any representations (one way or the other) as to third party rights in
> the data. As an illustration: you could dump all of OSM in to wikidata and
> the WMF would not need to change or do anything.
>

But the same works in reverse, doesn't it?  Wikidata project, just like WP
and OSM, is user contributable. If a user uploads data that violates
project's license, it should be deleted. And for that reason, both Wikidata
and OSM state the license under which the data is contributed and shared.
If I make an edit to OSM by copying data from Google, wouldn't that be the
same thing?

> (CC0), but the reverse depends on if the OSM contributor agreed to
> dedicate their edits to public domain.
>
> There is not really a practical and meaningful way in which an OSM
> contributor could do that, outside of facts that they have surveyed
> themselves and kept separate.
>

How I hate to diverge from the main topic, but alas... :)  This does sound
like a severe problem (that should be taken to a separate thread) - if I,
as a user, set the Public Domain checkbox, my assumptions are that my
contributions are PD. If I trace something based on some image data, I need
to specify that source, otherwise I am in violation of the source's
license. If I did not specify the source, and I checked the PD box, it can
be assumed that I am donating under PD. If this is not the case, it is a
violation of my contributor's rights - because otherwise my intention is
not being honored (i want other people to be able to use my work
unrestricted).  If anyone wants to comment, please start a new thread :)

>
> Without it, OSM data is licensed under ODbL, and cannot be copied. We
> should make it easier to detect what piece of OSM data is in PD.  I do like
> your USB analogy :) About names - you will be surprised to discover that MB
> and other places are actively pursuing Wikidata integration because WD
> tends to have a huge names list, possibly bigger than OSM itself?
>
>
> That is nice for MB, but problematic in more than one way for OSM.
>
Please elaborate, I know of at least one more company that is actively
doing that.   Sigh, another side topic :D

On Wed, Sep 20, 2017 at 1:58 PM, Simon Poole  wrote:

> [turning on broken record mode :-)]
> On 20.09.2017 17:54, Yuri Astrakhan wrote:
>
>
>
> * Oleksiy, OSM can use any data from Wikidata because of the public domain
> dedication
>
> While the WMF does not claim any rights in wikidata contents, it does not
> make any representations (one way or the other) as to third party rights in
> the data. As an illustration: you could dump all of OSM in to wikidata and
> the WMF would not need to change or do anything.
>
> (CC0), but the reverse depends on if the OSM contributor agreed to
> dedicate their edits to public domain.
>
> There is not really a practical and meaningful way in which an OSM
> contributor could do that, outside of facts that they have surveyed
> themselves and kept separate.
>
> Without it, OSM data is licensed under ODbL, and cannot be copied. We
> should make it easier to detect what piece of OSM data is in PD.  I do like
> your USB analogy :) About names - you will be surprised to discover that MB
> and other places are actively pursuing Wikidata integration because WD
> tends to have a huge names list, possibly bigger than OSM itself?
>
>
> That is nice for MB, but problematic in more than one way for OSM.
>
> Simon
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Simon Poole

[turning on broken record mode :-)]

On 20.09.2017 17:54, Yuri Astrakhan wrote:
>
>
> * Oleksiy, OSM can use any data from Wikidata because of the public
> domain dedication
While the WMF does not claim any rights in wikidata contents, it does
not make any representations (one way or the other) as to third party
rights in the data. As an illustration: you could dump all of OSM in to
wikidata and the WMF would not need to change or do anything.
> (CC0), but the reverse depends on if the OSM contributor agreed to
> dedicate their edits to public domain.
There is not really a practical and meaningful way in which an OSM
contributor could do that, outside of facts that they have surveyed
themselves and kept separate.

> Without it, OSM data is licensed under ODbL, and cannot be copied. We
> should make it easier to detect what piece of OSM data is in PD.  I do
> like your USB analogy :) About names - you will be surprised to
> discover that MB and other places are actively pursuing Wikidata
> integration because WD tends to have a huge names list, possibly
> bigger than OSM itself?
>
>
That is nice for MB, but problematic in more than one way for OSM.

Simon

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Oleksiy Muzalyev


On 20.09.17 18:35, Yuri Astrakhan wrote:
One other thing: lets not build walls between different projects. I 
know its in a human nature to do that, but lets not.  In Wikipedia, 
every language is also a separate project, and there I also saw a lot 
of "this is not how we do things around here".


[...]


Here is how the same idea was expressed in a classical poem, probably 
one of the best written in English: https://youtu.be/DJBLG0FvFMM


The poem ends in lines:

‘Men work together,’ I told him from the heart,
‘Whether they work together or apart.’

brgds

O.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Stefano

2017-09-20 19:07 GMT+02:00 Christoph Hormann :

>
>
> Don't assume such cases are just a freak anomaly - they are not.  OSM
> and wikidata are two very different projects which developed in very
> different contexts.  Just another example: For most cities and larger
> towns (at least in Germany) there exists an admin_level 6/8 unit with
> the same name and most of these seem to have a single wikidata item
> while in OSM we have two separate concepts for the populated place
> (place=city/town) and the administrative unit (boundary relation with
> boundary=administrative + admin_level=6/8).
>
> Hello Christoph,
this specific case was pointed out some time ago also on the italian
mailing list.
You're right on the issue, but on Wikidata recently someone created the
entity for the proper cities in Italy.
Here's the sparql query (all the objects which are marked capital of an
italian municipality)

http://tinyurl.com/yb2cjbae

Not all the municipalities have this property (6k objects instead of 8k)

> --
> Christoph Hormann
> http://www.imagico.de/
>

Ciao,
Stefano

>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Wednesday 20 September 2017, Yuri Astrakhan wrote:
>
> https://www.wikidata.org/wiki/Wikidata:WikiProject_Municipalities_of_
>Germany
>
> I think if you suggest it there, they will be happy to add it,
> allowing OSM objects to be properly tagged. Or just contribute there
> :)

But i don't think there is anything wrong with how wikidata represents 
things - for Hamburg wikidata for example has a single item:

https://www.wikidata.org/wiki/Q1055

which represents both the administrative unit and the populated place - 
which is perfectly fine.  And OSM does differentiate between them which 
is also fine.

Why should i attempt to change their data model to be identical to that 
of OSM (which would be a hopeless endeavor anyway because how things 
are represented in OSM is constantly changing)?

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

Also, there is a general country subdivision project with plenty of
information and current status.  I'm pretty sure OSM community has a lot of
good info to share:

https://www.wikidata.org/wiki/Wikidata:WikiProject_Country_subdivision

On Wed, Sep 20, 2017 at 1:28 PM, Yuri Astrakhan 
wrote:

> Don't assume such cases are just a freak anomaly - they are not.  OSM
>> and wikidata are two very different projects which developed in very
>> different contexts.  Just another example: For most cities and larger
>> towns (at least in Germany) there exists an admin_level 6/8 unit with
>> the same name and most of these seem to have a single wikidata item
>> while in OSM we have two separate concepts for the populated place
>> (place=city/town) and the administrative unit (boundary relation with
>> boundary=administrative + admin_level=6/8).
>>
>> Christoph, Wikidata community has a project
>
> https://www.wikidata.org/wiki/Wikidata:WikiProject_
> Municipalities_of_Germany
>
> I think if you suggest it there, they will be happy to add it, allowing
> OSM objects to be properly tagged. Or just contribute there :)
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

>
> Don't assume such cases are just a freak anomaly - they are not.  OSM
> and wikidata are two very different projects which developed in very
> different contexts.  Just another example: For most cities and larger
> towns (at least in Germany) there exists an admin_level 6/8 unit with
> the same name and most of these seem to have a single wikidata item
> while in OSM we have two separate concepts for the populated place
> (place=city/town) and the administrative unit (boundary relation with
> boundary=administrative + admin_level=6/8).
>
> Christoph, Wikidata community has a project

https://www.wikidata.org/wiki/Wikidata:WikiProject_Municipalities_of_Germany

I think if you suggest it there, they will be happy to add it, allowing OSM
objects to be properly tagged. Or just contribute there :)
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Wednesday 20 September 2017, Yuri Astrakhan wrote:
>
> Christoph, a valid point. Yet the duplicate would allow finding many
> of these errors, rather than leaving wp-only to go bad due to
> changing nature of the WP articles.

Actually no - you can find the errors just as well without adding the 
wikidata tags to OSM as after doing so.

The perpetuation of errors is one of the primary reasons why mechanical 
edits are often not considered favorably in OSM.

> As for sameness argument - lets 
> try to work on them on a case-by-case basis.

I don't really want to argue this here - as said i have no objections 
against having wikipedia/wikidata tags as references for 'related 
features' but many treat these references under the assumption that 
they indicate identical real world concepts exist on both ends - or 
even worth: they might think a lack of identity is an indication for a 
factual error in the data on one side.  I wanted to point out that this 
is a fundamentally incorrect assumption.

Don't assume such cases are just a freak anomaly - they are not.  OSM 
and wikidata are two very different projects which developed in very 
different contexts.  Just another example: For most cities and larger 
towns (at least in Germany) there exists an admin_level 6/8 unit with 
the same name and most of these seem to have a single wikidata item 
while in OSM we have two separate concepts for the populated place 
(place=city/town) and the administrative unit (boundary relation with 
boundary=administrative + admin_level=6/8).

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

One other thing: lets not build walls between different projects. I know
its in a human nature to do that, but lets not.  In Wikipedia, every
language is also a separate project, and there I also saw a lot of "this is
not how we do things around here".

Each project is ran by people.  Most people contribute to more than one
project, so lets not say "they do X, but we do Y", because often "they and
us" is the same person. Obviously some rules differ, and we should respect
that, but (I hope) most of us dedicate our volunteer time because we
believe in the general principal:  making knowledge available to everyone
freely.  OSM concentrates on geographical knowledge. Wikidata - on
classification.  Wikipedia - on descriptions.  All three, as a sum, can be
much greater than each one separately.  Lets keep that in mind when we
think how we can coexist better, and how to reduce the overlap.  Keeping
duplicates in sync is always harder than to let the tools do their data
merging work if needed.

On Wed, Sep 20, 2017 at 12:18 PM, Yuri Astrakhan 
wrote:

> Tobias, agree 100%, thanks.
>
> On Wed, Sep 20, 2017 at 12:14 PM, Tobias Knerr 
> wrote:
>
>> On 20.09.2017 17:02, Christoph Hormann wrote:
>> > It is best to regard the wikidata and wikipedia tags in OSM as 'related
>> > features' rather than identical objects.
>>
>> We shouldn't dilute the definition of the key because some incorrect
>> links exist in the database. If there's no 1:1 relationship between OSM
>> element and Wikidata item, and this cannot be fixed by editing Wikidata,
>> then no wikidata tag should be added.
>>
>> > These provide useful sources
>> > to research additional information (in particular wikipedia articles
>> > often link to additional sources)
>>
>> Sure, but the wikidata key is for "the Wikidata item about the feature",
>> not any related content that may be interesting.
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

Tobias, agree 100%, thanks.

On Wed, Sep 20, 2017 at 12:14 PM, Tobias Knerr  wrote:

> On 20.09.2017 17:02, Christoph Hormann wrote:
> > It is best to regard the wikidata and wikipedia tags in OSM as 'related
> > features' rather than identical objects.
>
> We shouldn't dilute the definition of the key because some incorrect
> links exist in the database. If there's no 1:1 relationship between OSM
> element and Wikidata item, and this cannot be fixed by editing Wikidata,
> then no wikidata tag should be added.
>
> > These provide useful sources
> > to research additional information (in particular wikipedia articles
> > often link to additional sources)
>
> Sure, but the wikidata key is for "the Wikidata item about the feature",
> not any related content that may be interesting.
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Tobias Knerr

On 20.09.2017 17:02, Christoph Hormann wrote:
> It is best to regard the wikidata and wikipedia tags in OSM as 'related 
> features' rather than identical objects.

We shouldn't dilute the definition of the key because some incorrect
links exist in the database. If there's no 1:1 relationship between OSM
element and Wikidata item, and this cannot be fixed by editing Wikidata,
then no wikidata tag should be added.

> These provide useful sources 
> to research additional information (in particular wikipedia articles 
> often link to additional sources)

Sure, but the wikidata key is for "the Wikidata item about the feature",
not any related content that may be interesting.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

>
> What will inevitably happen if you automatically add wikidata tags is
> that existing errors in either OSM (in form of incorrect wikipedia
> tags) or in wikidata (in form of incorrect connections to wikipedia
> articles) will get duplicated.
>

Christoph, a valid point. Yet the duplicate would allow finding many of
these errors, rather than leaving wp-only to go bad due to changing nature
of the WP articles. As for sameness argument - lets try to work on them on
a case-by-case basis. The vast majority of concepts are "good enough" - if
a park is tagged with the wikidata id for that park, and someone extends it
to add a few more trees, its not a big problem. If that edit combines two
parks into one, eventually it would get fixed, with two parks being
created.  And no, we won't be able to solve every edge case, but we will
solve it for the vast majority of them. After all, a map is an
approximation of the real world, not a perfect replica.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Wednesday 20 September 2017, Eugene Alvin Villar wrote:
>
> This is most certainly a wrong modeling in Wikidata. While we can
> just have one well-written and comprehensive Wikipedia article about
> the country/archipelago, in Wikidata, one item should correspond to
> one concept.

Maybe - but even if that is the case the wikidata concept of a country 
or archipelago is not necessarily the same as in OSM.  For countries 
and archipelagos this might sound strange (an archipelago is an 
archipelago, right?) but as you surely know the meaning of tags in OSM 
can be quite peculiar in the way it develops over time based on mapping 
needs and it would be quite insane if wikidata copied all these 
peculiarities in their classification system.

I don't know a lot about wikidata but as far as i can see every 
wikipedia article links to exactly one wikidata item and there are many 
geographic wikipedia articles that describe several different concepts 
together for which separate OSM features exist.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

On Wednesday 20 September 2017, Frederik Ramm wrote:
>
> Does that mean that the original plan of "fetching" (dare I say
> importing) 200k Wikidata links through automated connections from
> existing Wikipedia links is (a) dangerous because you can easily
> obtain a reference to something totally different, or (b) no problem
> because it is *to be expected* that an object's Wikipedia and
> Wikidata links point to different things an hence the import wouldn't
> introduce "errors" per se?

It all depends on how you use the data.  I think adding the wikidata 
tags is fine *because* i regard them as simple references to related 
features but if you'd insist on the idea that the OSM feature and the 
wikidata item refer to the same real world feature then inferring such 
an identity from an existing wikipedia tag is even more problematic - 
because the wikipedia tag was almost certainly not originally verified 
to refer to exactly the same concept as the OSM object.

Also keep in mind that both the OSM features and the wikidata items 
evolve over time and not every edit made in OSM (like extending the 
area of a forest polygon to include some additional tree covered area) 
is necessarily verified to still justify having the wikidata reference.

What will inevitably happen if you automatically add wikidata tags is 
that existing errors in either OSM (in form of incorrect wikipedia 
tags) or in wikidata (in form of incorrect connections to wikipedia 
articles) will get duplicated.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

Such an awesome discussion, thanks!

* https://www.wikidata.org/wiki/Special:GoToLinkedPage can already be used
to open a Wikipedia page when you only have a Wikidata ID.  It even accepts
a list of wiki sites. For example, this link automatically opens the wiki
page for Q3669 in the first available language ("pt" in this case)

https://www.wikidata.org/wiki/Special:GoToLinkedPage?itemid=Q3669=enwiki,ptwiki

* Sarah, thanks for the heads up about Nominatim using Wikipedia tags.  I
recently added page popularity (pageviews) to the OSM+Wikidata service.
Another metric is the number of Wikipedia articles in different languages
per topic (sitelinks count).  Together, they can be used to calculate
relative weights.

* I am a bit radical, but not enough to propose we get rid wikipedia tags
just yet.  They sometimes provide a good indication of the original
intent.  Once Wikidata is used in all the tooling, we may revisit, but not
until then.  But yes, wikipedia tags are very unstable, especially when
articles get renamed because multiple places have identical names, thus
creating a link to disambig. So in general, they often go stale and become
less useful without any indication.

* Oleksiy, OSM can use any data from Wikidata because of the public domain
dedication (CC0), but the reverse depends on if the OSM contributor agreed
to dedicate their edits to public domain. Without it, OSM data is licensed
under ODbL, and cannot be copied. We should make it easier to detect what
piece of OSM data is in PD.  I do like your USB analogy :) About names -
you will be surprised to discover that MB and other places are actively
pursuing Wikidata integration because WD tends to have a huge names list,
possibly bigger than OSM itself?

* Christoph, a very valid point in general. Do you have any statistics on
how often multiple meanings per osm object is a problem? In my experience,
this is very rare, but hard to say without numbers.  For the case of the
island being both a country and a land feature, I think it would benefit
OSM to actually have two objects with the same geometry - e.g. two
relations containing the same way(s).  One relation would treat it as an
admin boundary, with all the related tags, the other - as a land feature.
Data consumers would treat them separately. Conflating tags related to both
concepts into one object is not very good.  In a more general terms, you
usually have three cases:
-- 1:1 (most common imo)
-- one osm obj being a part of larger page (e.g. a list of churches). I
don't think wikidata/wikipedia tag is appropriate in this case, as that
page is not about this specific object, but about a class of similar
objects. We could use listed-on:wp, or partof:wp, or some other tag.
-- Your case - multiple concepts for the same object. Use either a
semicolon separated list of wd ids, or (better) - create multiple relations
to describe multiple concepts.

* Frederik, that bit of a small personal attack is uncalled for. I exposed
a lot of existing bad data, not added it. And I created complex tooling to
help everyone resolve it as a community, rather than try to tackle all of
it by myself.  A system for fixing problems is always better than one
person doing it by hand, and later retiring because the challenge is too
great.  Also, corresponding wikidata tag is not a bad data - it is simply a
copy of the existing Wikipedia tag, making it easier for tools and humans
to find and fix. As for your last email - fetching *corresponding* wikidata
items is not an error - its a duplicate of an existing information. That
information might be incomplete, but that's a separate issue.

* Lester, I'm not sure I understood your Douglas Adams example, PM me and
lets try to figure it out. It might has to do with ranking of each statement

See also:
Feature request for any lang fallback:
https://phabricator.wikimedia.org/T176321
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Eugene Alvin Villar

On Wed, Sep 20, 2017 at 11:02 PM, Christoph Hormann  wrote:

> Simple example:  The Faroe Islands are both a country:
>
> http://www.openstreetmap.org/relation/52939
>
> and an archipelago:
>
> http://www.openstreetmap.org/relation/3067431
>
> in OSM which are represented as separate features obviously.  Both
> reference the same wikidata item [...]
>

This is most certainly a wrong modeling in Wikidata. While we can just have
one well-written and comprehensive Wikipedia article about the
country/archipelago, in Wikidata, one item should correspond to one
concept. So there should be separate Wikidata items for the archipelago and
the country. The fact that it isn't like that right now is simply because
Wikidata is an ongoing project, just like OSM.

As an example, we have the island of Bali (
https://www.wikidata.org/wiki/Q4648) versus the province of Bali (
https://www.wikidata.org/wiki/Q3125978), though there is only just one
English Wikipedia article covering both concepts:
https://en.wikipedia.org/wiki/Bali
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag

2017-09-20 Thread Frederik Ramm

Hi,

On 20.09.2017 17:02, Christoph Hormann wrote:
> It is best to regard the wikidata and wikipedia tags in OSM as 'related 
> features' rather than identical objects.  These provide useful sources 
> to research additional information (in particular wikipedia articles 
> often link to additional sources) but you should never try to fix or 
> add something in OSM - be that a name tag or coordinates - based purely 
> on the assumption that the wikidata object referenced via tag is the 
> same as the OSM feature.

Does that mean that the original plan of "fetching" (dare I say
importing) 200k Wikidata links through automated connections from
existing Wikipedia links is (a) dangerous because you can easily obtain
a reference to something totally different, or (b) no problem because it
is *to be expected* that an object's Wikipedia and Wikidata links point
to different things an hence the import wouldn't introduce "errors" per se?

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag