Re: [OSM-talk] Fixing OSM wikipedia redirects
On Mon, 2017-09-25 at 23:11 -0400, Yuri Astrakhan wrote: > Fixing them by hand seems like a huge waste of the > community time I agree. This is purely mechanical work. Drudgery is evil. I'm willing to try writing a script once I finish my other scripting projects. This should be straightforward. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
On Tue, Sep 26, 2017 at 04:03:35AM -0400, Yuri Astrakhan wrote: > Sarah, my understanding is that MapRoulette does not support it -- I cannot > upload the following: > > For Object ID , change one set of tags for another -- accept or > decline? No you can't and you shouldn't. Clicking through to two websites and copying the link is absolutely ok. > Yes, the community can do it, the question is - should it? Given 2 > challenges, one that requires some thought, and the other that require > clicking yes without thinking, shouldn't we opt to the one that computers > cannot do? That's exactly what I'm saying. People don't always want to do the complicated tasks. There is enough in that in the day job. Sometimes just clicking through a pile of wikipedia links is totally enough. Don't just leave the complicated, messy parts of your scripts to the mappers. It tends to be demotivating because there is no real progress if you always have to do a lot of research for each single item. And it is especially demotivating when it is the result of an automated edit because it feels like spending your time to clean up other people's mess. We've just cleaned up hundereds of thousands of polygons through Maproulette tasks. The popular challenges were the ones which were it was clear what was to do (not necessarily the same as fixing was easy, there were complicated multipolygon relations involved), not the ones where you had to carefully analyse the map to understand what is going on. Sarah > We seemed to assume that donated time is free, unlimited, and > has very little value. I feel we should treat donated time as the most > precious and most scarce resource we could get. > > On Tue, Sep 26, 2017 at 3:48 AM, Sarah Hoffmann wrote: > > > On Mon, Sep 25, 2017 at 11:53:03PM -0400, Yuri Astrakhan wrote: > > > According to Martijn (of MapRoulette fame), there is no way a challenge > > can > > > link to object IDs. MapRoulette can only highlight location. Nor can I > > > provide a proposed fix, which means someone would have to manually find > > the > > > broken object, navigate to Wikipedia, copy/paste the title, and save the > > > object. I guesstimate 1 minute per object on average... that's nearly > > 700 > > > hours of community time - a huge waste of human brain power that could be > > > spent on a much more challenging and less automatable tasks. > > > > We'd have 40.000 more recently reviewed objects in the database. Given how > > much the quality of the OSM data decays with time, I would consider that > > a welcome boost to overall quality. > > > > And my experience with the OSM community is that there are a lot of people > > who wouldn't consider such a task a waste of time but as a wonderful > > opportunity to relax in the evening. Maproulette has the advantage that > > you can just click away and do one object after the next. I would recommend > > to break the 40.000 objects into local batches of 1000 or 2000 and just > > load it into Maproulette. Add step-by-step isntructions how to fix the > > links and I'm sure you'll be surprised how quickly everything is done. > > > > Kind regards > > > > Sarah > > > > > > > > Osmose might be a good alternative, and might even lower the total number > > > of hours required, but still - would that significantly benefit the > > > project? These tags are just a tiny arbitrary subset of one million > > > wikipedia-tagged objects. Verifying just them by hand seems like a waste > > > of human intelligence. Instead, we can run queries to produce knowingly > > bad > > > objects and let community fix those. I hope we can let machines do > > mindless > > > tasks, and let humans do decision making. This would improve > > contributors > > > morale, instead of making them feel like robots :) > > > > > > Clarifying: the OSM objects already point to those pages via redirect. > > The > > > redirect information is only stored in Wikipedia. > > > > > > On Mon, Sep 25, 2017 at 11:18 PM, Marc Gemis > > wrote: > > > > > > > or via Osmose ? > > > > > > > > On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis > > wrote: > > > > > what about a Maproulette task ? > > > > > > > > > > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan < > > yuriastrak...@gmail.com> > > > > wrote: > > > > >> At the moment, there are nearly 40,000 OSM objects whose wikipedia > > tag > > > > does > > > > >> not match their wikidata tag. Most of them are Wikipedia redirects, > > > > whose > > > > >> target is the right wikipedia article. If we are not ready to > > abandon > > > > >> wikipedia tags just yet (I don't think we should ATM), I think we > > > > should fix > > > > >> them. Fixing them by hand seems like a huge waste of the community > > > > time, > > > > >> when it can be semi-automated. > > > > >> > > > > >> I propose that a small program, possibly a plugin to JOSM, would > > change > > > > >> wikipedia tags to point to the target article instead of the > > redirect. > > > > >> > > > > >> Thoughts? > >
Re: [OSM-talk] Fixing OSM wikipedia redirects
Sarah, my understanding is that MapRoulette does not support it -- I cannot upload the following: For Object ID , change one set of tags for another -- accept or decline? Would be great if I was wrong. Yes, the community can do it, the question is - should it? Given 2 challenges, one that requires some thought, and the other that require clicking yes without thinking, shouldn't we opt to the one that computers cannot do? We seemed to assume that donated time is free, unlimited, and has very little value. I feel we should treat donated time as the most precious and most scarce resource we could get. On Tue, Sep 26, 2017 at 3:48 AM, Sarah Hoffmann wrote: > On Mon, Sep 25, 2017 at 11:53:03PM -0400, Yuri Astrakhan wrote: > > According to Martijn (of MapRoulette fame), there is no way a challenge > can > > link to object IDs. MapRoulette can only highlight location. Nor can I > > provide a proposed fix, which means someone would have to manually find > the > > broken object, navigate to Wikipedia, copy/paste the title, and save the > > object. I guesstimate 1 minute per object on average... that's nearly > 700 > > hours of community time - a huge waste of human brain power that could be > > spent on a much more challenging and less automatable tasks. > > We'd have 40.000 more recently reviewed objects in the database. Given how > much the quality of the OSM data decays with time, I would consider that > a welcome boost to overall quality. > > And my experience with the OSM community is that there are a lot of people > who wouldn't consider such a task a waste of time but as a wonderful > opportunity to relax in the evening. Maproulette has the advantage that > you can just click away and do one object after the next. I would recommend > to break the 40.000 objects into local batches of 1000 or 2000 and just > load it into Maproulette. Add step-by-step isntructions how to fix the > links and I'm sure you'll be surprised how quickly everything is done. > > Kind regards > > Sarah > > > > > Osmose might be a good alternative, and might even lower the total number > > of hours required, but still - would that significantly benefit the > > project? These tags are just a tiny arbitrary subset of one million > > wikipedia-tagged objects. Verifying just them by hand seems like a waste > > of human intelligence. Instead, we can run queries to produce knowingly > bad > > objects and let community fix those. I hope we can let machines do > mindless > > tasks, and let humans do decision making. This would improve > contributors > > morale, instead of making them feel like robots :) > > > > Clarifying: the OSM objects already point to those pages via redirect. > The > > redirect information is only stored in Wikipedia. > > > > On Mon, Sep 25, 2017 at 11:18 PM, Marc Gemis > wrote: > > > > > or via Osmose ? > > > > > > On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis > wrote: > > > > what about a Maproulette task ? > > > > > > > > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan < > yuriastrak...@gmail.com> > > > wrote: > > > >> At the moment, there are nearly 40,000 OSM objects whose wikipedia > tag > > > does > > > >> not match their wikidata tag. Most of them are Wikipedia redirects, > > > whose > > > >> target is the right wikipedia article. If we are not ready to > abandon > > > >> wikipedia tags just yet (I don't think we should ATM), I think we > > > should fix > > > >> them. Fixing them by hand seems like a huge waste of the community > > > time, > > > >> when it can be semi-automated. > > > >> > > > >> I propose that a small program, possibly a plugin to JOSM, would > change > > > >> wikipedia tags to point to the target article instead of the > redirect. > > > >> > > > >> Thoughts? > > > >> > > > >> ___ > > > >> talk mailing list > > > >> talk@openstreetmap.org > > > >> https://lists.openstreetmap.org/listinfo/talk > > > >> > > > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
On Mon, Sep 25, 2017 at 11:53:03PM -0400, Yuri Astrakhan wrote: > According to Martijn (of MapRoulette fame), there is no way a challenge can > link to object IDs. MapRoulette can only highlight location. Nor can I > provide a proposed fix, which means someone would have to manually find the > broken object, navigate to Wikipedia, copy/paste the title, and save the > object. I guesstimate 1 minute per object on average... that's nearly 700 > hours of community time - a huge waste of human brain power that could be > spent on a much more challenging and less automatable tasks. We'd have 40.000 more recently reviewed objects in the database. Given how much the quality of the OSM data decays with time, I would consider that a welcome boost to overall quality. And my experience with the OSM community is that there are a lot of people who wouldn't consider such a task a waste of time but as a wonderful opportunity to relax in the evening. Maproulette has the advantage that you can just click away and do one object after the next. I would recommend to break the 40.000 objects into local batches of 1000 or 2000 and just load it into Maproulette. Add step-by-step isntructions how to fix the links and I'm sure you'll be surprised how quickly everything is done. Kind regards Sarah > > Osmose might be a good alternative, and might even lower the total number > of hours required, but still - would that significantly benefit the > project? These tags are just a tiny arbitrary subset of one million > wikipedia-tagged objects. Verifying just them by hand seems like a waste > of human intelligence. Instead, we can run queries to produce knowingly bad > objects and let community fix those. I hope we can let machines do mindless > tasks, and let humans do decision making. This would improve contributors > morale, instead of making them feel like robots :) > > Clarifying: the OSM objects already point to those pages via redirect. The > redirect information is only stored in Wikipedia. > > On Mon, Sep 25, 2017 at 11:18 PM, Marc Gemis wrote: > > > or via Osmose ? > > > > On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis wrote: > > > what about a Maproulette task ? > > > > > > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan > > wrote: > > >> At the moment, there are nearly 40,000 OSM objects whose wikipedia tag > > does > > >> not match their wikidata tag. Most of them are Wikipedia redirects, > > whose > > >> target is the right wikipedia article. If we are not ready to abandon > > >> wikipedia tags just yet (I don't think we should ATM), I think we > > should fix > > >> them. Fixing them by hand seems like a huge waste of the community > > time, > > >> when it can be semi-automated. > > >> > > >> I propose that a small program, possibly a plugin to JOSM, would change > > >> wikipedia tags to point to the target article instead of the redirect. > > >> > > >> Thoughts? > > >> > > >> ___ > > >> talk mailing list > > >> talk@openstreetmap.org > > >> https://lists.openstreetmap.org/listinfo/talk > > >> > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
Mark, these do rarely happen, but do they add value? Having data that points to a non-(machine)-verifiable redirect, a redirect that could be deleted or changed at any moment is very fragile. I think these should be resolved to the target, and treated as I described in [1] - links to wikipedia pages about multiple objects. This way it will be clear that the tag points to a subsection/component of an article, and will be treated accordingly. [1] https://wiki.openstreetmap.org/wiki/Wikipedia_Improvement_Tasks#Links_to_Wikipedia_pages_about_multiple_objects On Tue, Sep 26, 2017 at 2:58 AM, Mark Wagner wrote: > On Mon, 25 Sep 2017 23:11:52 -0400 > Yuri Astrakhan wrote: > > > At the moment, there are nearly 40,000 OSM objects whose wikipedia > > tag does not match their wikidata tag. Most of them are Wikipedia > > redirects, whose target is the right wikipedia article. > > What about the ones where the article is a Wikipedia redirect, whose > target is *almost, but not quite* the right Wikipedia article? I'm > thinking about things like neighborhoods of a city, where Wikipedia > currently has a redirect to the city article. > > -- > Mark > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
On Mon, 25 Sep 2017 23:11:52 -0400 Yuri Astrakhan wrote: > At the moment, there are nearly 40,000 OSM objects whose wikipedia > tag does not match their wikidata tag. Most of them are Wikipedia > redirects, whose target is the right wikipedia article. What about the ones where the article is a Wikipedia redirect, whose target is *almost, but not quite* the right Wikipedia article? I'm thinking about things like neighborhoods of a city, where Wikipedia currently has a redirect to the city article. -- Mark ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
You do have a valid point about getting the local community exposure to Wikidata. But I see no contradiction between that and my proposal, because I think it would be very easy to come up with countless Wikipedia/Wikidata cleanup tasks that require human attention. There is always be plenty of work. After my program runs, there will be thousands of items that could not be resolved as easily. For example, there will be tons of cases when wikipedia and wikidata point to different entities. Some of them are legit - e.g. island (wikipedia) vs administrative area (wikidata). Redirect resolution would not introduce communities to wikidata, but rather teach community how to mindlessly click "accept", and I would much rather avoid that - as this might result in bigger problems when a real decision needs to be made. On Tue, Sep 26, 2017 at 1:04 AM, Marc Gemis wrote: > By using Osmose, it would be possible to involve the local > communities. People would learn about Wikidata, and might start adding > them to other objects as well. They might even start contributing to > Wikidata as well. > By just running your program, you would only fix a small number of > entries and nobody would know, nobody would bother about them. > > I have the feeling that a program can fix some errors in a short > period, but doesn't bring anything else. Allowing people to fix > trivial problems, allow them to get familiar with the data, they will > take some form of ownership and maintain the data and that is more > beneficial in the long term than an automated quick fix now. > > m. > > On Tue, Sep 26, 2017 at 5:53 AM, Yuri Astrakhan > wrote: > > According to Martijn (of MapRoulette fame), there is no way a challenge > can > > link to object IDs. MapRoulette can only highlight location. Nor can I > > provide a proposed fix, which means someone would have to manually find > the > > broken object, navigate to Wikipedia, copy/paste the title, and save the > > object. I guesstimate 1 minute per object on average... that's nearly > 700 > > hours of community time - a huge waste of human brain power that could be > > spent on a much more challenging and less automatable tasks. > > > > Osmose might be a good alternative, and might even lower the total > number of > > hours required, but still - would that significantly benefit the project? > > These tags are just a tiny arbitrary subset of one million > wikipedia-tagged > > objects. Verifying just them by hand seems like a waste of human > > intelligence. Instead, we can run queries to produce knowingly bad > objects > > and let community fix those. I hope we can let machines do mindless > tasks, > > and let humans do decision making. This would improve contributors > morale, > > instead of making them feel like robots :) > > > > Clarifying: the OSM objects already point to those pages via redirect. > The > > redirect information is only stored in Wikipedia. > > > > On Mon, Sep 25, 2017 at 11:18 PM, Marc Gemis > wrote: > >> > >> or via Osmose ? > >> > >> On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis > wrote: > >> > what about a Maproulette task ? > >> > > >> > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan > >> > wrote: > >> >> At the moment, there are nearly 40,000 OSM objects whose wikipedia > tag > >> >> does > >> >> not match their wikidata tag. Most of them are Wikipedia redirects, > >> >> whose > >> >> target is the right wikipedia article. If we are not ready to abandon > >> >> wikipedia tags just yet (I don't think we should ATM), I think we > >> >> should fix > >> >> them. Fixing them by hand seems like a huge waste of the community > >> >> time, > >> >> when it can be semi-automated. > >> >> > >> >> I propose that a small program, possibly a plugin to JOSM, would > change > >> >> wikipedia tags to point to the target article instead of the > redirect. > >> >> > >> >> Thoughts? > >> >> > >> >> ___ > >> >> talk mailing list > >> >> talk@openstreetmap.org > >> >> https://lists.openstreetmap.org/listinfo/talk > >> >> > > > > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
By using Osmose, it would be possible to involve the local communities. People would learn about Wikidata, and might start adding them to other objects as well. They might even start contributing to Wikidata as well. By just running your program, you would only fix a small number of entries and nobody would know, nobody would bother about them. I have the feeling that a program can fix some errors in a short period, but doesn't bring anything else. Allowing people to fix trivial problems, allow them to get familiar with the data, they will take some form of ownership and maintain the data and that is more beneficial in the long term than an automated quick fix now. m. On Tue, Sep 26, 2017 at 5:53 AM, Yuri Astrakhan wrote: > According to Martijn (of MapRoulette fame), there is no way a challenge can > link to object IDs. MapRoulette can only highlight location. Nor can I > provide a proposed fix, which means someone would have to manually find the > broken object, navigate to Wikipedia, copy/paste the title, and save the > object. I guesstimate 1 minute per object on average... that's nearly 700 > hours of community time - a huge waste of human brain power that could be > spent on a much more challenging and less automatable tasks. > > Osmose might be a good alternative, and might even lower the total number of > hours required, but still - would that significantly benefit the project? > These tags are just a tiny arbitrary subset of one million wikipedia-tagged > objects. Verifying just them by hand seems like a waste of human > intelligence. Instead, we can run queries to produce knowingly bad objects > and let community fix those. I hope we can let machines do mindless tasks, > and let humans do decision making. This would improve contributors morale, > instead of making them feel like robots :) > > Clarifying: the OSM objects already point to those pages via redirect. The > redirect information is only stored in Wikipedia. > > On Mon, Sep 25, 2017 at 11:18 PM, Marc Gemis wrote: >> >> or via Osmose ? >> >> On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis wrote: >> > what about a Maproulette task ? >> > >> > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan >> > wrote: >> >> At the moment, there are nearly 40,000 OSM objects whose wikipedia tag >> >> does >> >> not match their wikidata tag. Most of them are Wikipedia redirects, >> >> whose >> >> target is the right wikipedia article. If we are not ready to abandon >> >> wikipedia tags just yet (I don't think we should ATM), I think we >> >> should fix >> >> them. Fixing them by hand seems like a huge waste of the community >> >> time, >> >> when it can be semi-automated. >> >> >> >> I propose that a small program, possibly a plugin to JOSM, would change >> >> wikipedia tags to point to the target article instead of the redirect. >> >> >> >> Thoughts? >> >> >> >> ___ >> >> talk mailing list >> >> talk@openstreetmap.org >> >> https://lists.openstreetmap.org/listinfo/talk >> >> > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
According to Martijn (of MapRoulette fame), there is no way a challenge can link to object IDs. MapRoulette can only highlight location. Nor can I provide a proposed fix, which means someone would have to manually find the broken object, navigate to Wikipedia, copy/paste the title, and save the object. I guesstimate 1 minute per object on average... that's nearly 700 hours of community time - a huge waste of human brain power that could be spent on a much more challenging and less automatable tasks. Osmose might be a good alternative, and might even lower the total number of hours required, but still - would that significantly benefit the project? These tags are just a tiny arbitrary subset of one million wikipedia-tagged objects. Verifying just them by hand seems like a waste of human intelligence. Instead, we can run queries to produce knowingly bad objects and let community fix those. I hope we can let machines do mindless tasks, and let humans do decision making. This would improve contributors morale, instead of making them feel like robots :) Clarifying: the OSM objects already point to those pages via redirect. The redirect information is only stored in Wikipedia. On Mon, Sep 25, 2017 at 11:18 PM, Marc Gemis wrote: > or via Osmose ? > > On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis wrote: > > what about a Maproulette task ? > > > > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan > wrote: > >> At the moment, there are nearly 40,000 OSM objects whose wikipedia tag > does > >> not match their wikidata tag. Most of them are Wikipedia redirects, > whose > >> target is the right wikipedia article. If we are not ready to abandon > >> wikipedia tags just yet (I don't think we should ATM), I think we > should fix > >> them. Fixing them by hand seems like a huge waste of the community > time, > >> when it can be semi-automated. > >> > >> I propose that a small program, possibly a plugin to JOSM, would change > >> wikipedia tags to point to the target article instead of the redirect. > >> > >> Thoughts? > >> > >> ___ > >> talk mailing list > >> talk@openstreetmap.org > >> https://lists.openstreetmap.org/listinfo/talk > >> > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
or via Osmose ? On Tue, Sep 26, 2017 at 5:16 AM, Marc Gemis wrote: > what about a Maproulette task ? > > On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan > wrote: >> At the moment, there are nearly 40,000 OSM objects whose wikipedia tag does >> not match their wikidata tag. Most of them are Wikipedia redirects, whose >> target is the right wikipedia article. If we are not ready to abandon >> wikipedia tags just yet (I don't think we should ATM), I think we should fix >> them. Fixing them by hand seems like a huge waste of the community time, >> when it can be semi-automated. >> >> I propose that a small program, possibly a plugin to JOSM, would change >> wikipedia tags to point to the target article instead of the redirect. >> >> Thoughts? >> >> ___ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk >> ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
what about a Maproulette task ? On Tue, Sep 26, 2017 at 5:11 AM, Yuri Astrakhan wrote: > At the moment, there are nearly 40,000 OSM objects whose wikipedia tag does > not match their wikidata tag. Most of them are Wikipedia redirects, whose > target is the right wikipedia article. If we are not ready to abandon > wikipedia tags just yet (I don't think we should ATM), I think we should fix > them. Fixing them by hand seems like a huge waste of the community time, > when it can be semi-automated. > > I propose that a small program, possibly a plugin to JOSM, would change > wikipedia tags to point to the target article instead of the redirect. > > Thoughts? > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk