Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Andrew Hain]

How much use are you making of tools developed for identifying contributions to 
be redacted in the licence change?

--
Andrew

From: Frederik Ramm <frede...@remote.org>
Sent: 28 August 2017 13:43:44
Cc: Talk Openstreetmap; talk...@openstreetmap.org
Subject: Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by 
user chdr

Hi,

On 08/27/2017 08:51 PM, Mikel Maron wrote:
> Also, Frederik, I think your script picked up false positives. Spot
> checked in DC, and these are expansions of both the street and the
> quadrant ("St NW" -> "Street Northwest"(. Can we fix the script and
> regen the list?

I have modified my "name equality rule" to consider "N" equal to "North"
etc., also it will ignore case, whitespace, and as before the usual
street type expansions (St->Street etc).

This brings the number of problematic objects down by around 5500, and
practically all of them are in the US. However, I noticed that I forgot
to account for "Saint"->"St", and will re-do the numbers yet again
before publishing an updated list.

I think the best course of action would be:

1. Wait a while, until various communities (potentially pointed to this
conversation via the widely-read weekly new roundup) have had the time
to check whether my automated assessment of which names count as
"contributed" by chdr is correct. Mikel has found the issue above and I
fixed it; it is quite possible that there are others.

2. Run the redaction, and remove all names contributed by chdr. At
present it looks as if less than 10% of these objects had a different
name before; more than 90% had not name at all. Perhaps it is indeed
best to remove the name in these cases as well instead of reverting to
the old name.

3. Load the IDs of all affected objects in a MapRoulette task or
similar, so people can check the names by survey, or from different
sources. (I assume that, as Simon pointed out, open data will not be
available for all countries affected. I fear that, with MapRoulette
geared towards armchair mapping, there might be a temptation for people
to yet again fill in the blanks from inadmissible sources. Maybe we
should limit the use of MapRoulette to countries where we know that open
sources exist, and use fixme tags or notes for other countries?)

I think that would be cleaner than verifying the names ahead of time.
Also it would create an audit trail - from the object history, you could
then see that the name was removed for copyright reasons, and you could
then see that user XYZ has added a new name. If it should later turn out
that this name was also copied from an indadmissible source, we know
that user XYZ is at fault, whereas people creating lists with
independently verified names is not something that would give us such a
recording.

I must apologize for not having given a time frame in my initial email;
there's absolutely no reason to panic. This matter has been sitting idle
for years, and a few more weeks won't kill us. We can sort this out
calmly and then do the right thing.

Bye
Frederik

--
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Frederik Ramm]

Hi,

On 08/27/2017 08:51 PM, Mikel Maron wrote:
> Also, Frederik, I think your script picked up false positives. Spot
> checked in DC, and these are expansions of both the street and the
> quadrant ("St NW" -> "Street Northwest"(. Can we fix the script and
> regen the list?

I have modified my "name equality rule" to consider "N" equal to "North"
etc., also it will ignore case, whitespace, and as before the usual
street type expansions (St->Street etc).

This brings the number of problematic objects down by around 5500, and
practically all of them are in the US. However, I noticed that I forgot
to account for "Saint"->"St", and will re-do the numbers yet again
before publishing an updated list.

I think the best course of action would be:

1. Wait a while, until various communities (potentially pointed to this
conversation via the widely-read weekly new roundup) have had the time
to check whether my automated assessment of which names count as
"contributed" by chdr is correct. Mikel has found the issue above and I
fixed it; it is quite possible that there are others.

2. Run the redaction, and remove all names contributed by chdr. At
present it looks as if less than 10% of these objects had a different
name before; more than 90% had not name at all. Perhaps it is indeed
best to remove the name in these cases as well instead of reverting to
the old name.

3. Load the IDs of all affected objects in a MapRoulette task or
similar, so people can check the names by survey, or from different
sources. (I assume that, as Simon pointed out, open data will not be
available for all countries affected. I fear that, with MapRoulette
geared towards armchair mapping, there might be a temptation for people
to yet again fill in the blanks from inadmissible sources. Maybe we
should limit the use of MapRoulette to countries where we know that open
sources exist, and use fixme tags or notes for other countries?)

I think that would be cleaner than verifying the names ahead of time.
Also it would create an audit trail - from the object history, you could
then see that the name was removed for copyright reasons, and you could
then see that user XYZ has added a new name. If it should later turn out
that this name was also copied from an indadmissible source, we know
that user XYZ is at fault, whereas people creating lists with
independently verified names is not something that would give us such a
recording.

I must apologize for not having given a time frame in my initial email;
there's absolutely no reason to panic. This matter has been sitting idle
for years, and a few more weeks won't kill us. We can sort this out
calmly and then do the right thing.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Andy Mabbett]

On 28 August 2017 at 08:57, Warin <61sundow...@gmail.com> wrote:

> This is a data deletion - a very different thing.

Another editor in this discussion says that they:

changed the name tag to chdr_USA_AZ_name_fixup_required

That's not merely a deletion.

It behoves DWG to adhere to their own policies for such things, even
if the deletions are inevitable.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Andy Mabbett]

On 28 August 2017 at 01:56, john whelan  wrote:

> It has been brought to the OSMF's attention.  It has been verified that
> there are "Easter Eggs" from Google are in there.

"Easter Eggs"? Do you perhaps mean "trap streets":

   https://en.wikipedia.org/wiki/Trap_street ?

You say that this has been verified, but I have not seen any alleged
examples - where are they?

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Maarten Deen]

On 2017-08-28 13:02, Christoph Hormann wrote:

On Monday 28 August 2017, Greg Morgan wrote:

We do get to go through the five stages.  We do get to express the
emotions until acceptance of our fate as part of the healing process.
 There is no rubber stamping!

https://en.wikipedia.org/wiki/K%C3%BCbler-Ross_model


You are free to express your emotions but I think you are overreaching
here with comparing a redaction of data in OSM to the grief over a
serious personal loss.


I'm quite sure Greg's comments were meant sarcasticly.

Maarten

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Mikel Maron]

The essential discussion here is -- OSM communities can put together plans 
ahead of this redaction, in order to minimize impact to the map. With 
sufficient legal process. This seems like a perfectly reasonable thing to plan 
out, rather than doing a large scale revert and scrambling to clean up after.
The chdr problem has been with us for years. There is little risk in giving 
slightly more time to plan ahead. Perhaps that is best done country by country. 
* Mikel Maron * +14152835207 @mikel s:mikelmaron 

On Monday, August 28, 2017 7:06 AM, Christoph Hormann  
wrote:
 

 On Monday 28 August 2017, Greg Morgan wrote:
> We do get to go through the five stages.  We do get to express the
> emotions until acceptance of our fate as part of the healing process.
>  There is no rubber stamping!
>
> https://en.wikipedia.org/wiki/K%C3%BCbler-Ross_model

You are free to express your emotions but I think you are overreaching 
here with comparing a redaction of data in OSM to the grief over a 
serious personal loss.

In case this was not clear the term 'healed' as i used it was referring 
to the legal concept of healing a breach of contract or other legal 
infractions by ceasing some activity or doing something you neglected 
to do before.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


   ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Christoph Hormann]

On Monday 28 August 2017, Greg Morgan wrote:
> We do get to go through the five stages.  We do get to express the
> emotions until acceptance of our fate as part of the healing process.
>  There is no rubber stamping!
>
> https://en.wikipedia.org/wiki/K%C3%BCbler-Ross_model

You are free to express your emotions but I think you are overreaching 
here with comparing a redaction of data in OSM to the grief over a 
serious personal loss.

In case this was not clear the term 'healed' as i used it was referring 
to the legal concept of healing a breach of contract or other legal 
infractions by ceasing some activity or doing something you neglected 
to do before.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Greg Morgan]

On Mon, Aug 28, 2017 at 1:39 AM, Christoph Hormann  wrote:

> In fact reading this thread i cannot really believe this discussion is
> actually happening, that there seem to be people who think that a large
> scale unauthorized use of data can be 'healed' by rubber stamping it
> with a 'review' of the names.
>

-1
We do get to go through the five stages.  We do get to express the emotions
until acceptance of our fate as part of the healing process.  There is no
rubber stamping!

https://en.wikipedia.org/wiki/K%C3%BCbler-Ross_model
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Greg Morgan]

On Mon, Aug 28, 2017 at 3:16 AM, James  wrote:

> As Stewart has pointed out there are some changes that are valid(name
> expansion). I think Mr.Ramm needs to revise his selection algorithm before
> mass deletion
>

I don't think that it make a difference.   I certainly don't want to make
more edits on questionable data!

My heart goes out to other countries without a source like to rebuild clean.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[James]

As Stewart has pointed out there are some changes that are valid(name
expansion). I think Mr.Ramm needs to revise his selection algorithm before
mass deletion

On Aug 28, 2017 6:05 AM, "joost schouppe"  wrote:

2017-08-28 10:27 GMT+02:00 Simon Poole :

> What surprises me most about the discussion up to now that it is
> centered around the impact on the US and Canada were the removals are
> rather small both relatively and absolute*, actually are more at the
> nuisance level than anything else, and can easily be added back, likely
> in a couple of days  from sources that are already pre-approved as
> reference material.
>
> Our concern should be more about Mexico, Brazil and other countries
> where it is at least not obvious to me if the local communities are
> aware of the issue and if we have any plan at all how we possibly could
> mitigate the impact.
>

I can't say I'm surprised by that myself. This is a conversation in
English, on a mailing list, bot things not very popular in Latin America.
I've posted this thread to the Latam telegram group, with an offer to
translate if needed.

-- 
Joost Schouppe
OpenStreetMap  |
Twitter  | LinkedIn
 | Meetup


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Greg Morgan]

On Sun, Aug 27, 2017 at 11:51 AM, Mikel Maron  wrote:

> > we can find a good workflow for that. I wasn't expecting the community
> to start working on this pre-redaction but if people prefer that to
> fixing issues later...
>
> Absolutely, let's do this!
>
>
On Sun, Aug 27, 2017 at 10:55 AM, Martijn van Exel  > wrote:
>
> Happy to help. All we'd need for MapRoulette is a list of locations and a
> proper description of the work we'd expect people to do. Anyone can create
> the challenge but I'd be happy to do it.
> Martijn
>
>
>
 Change set # 51502902

Here's what I did and the plan:

* I selected out Arizona USA.
* I used the magic of regular expressions to change the way id in the file
Frederik provided to a josm remote control command.
* The remote control command allowed me to load all the data into one
file.  I changed the name tag to chdr_USA_AZ_name_fixup_required.  The new
name creates a great search target.
* I made no other changes before uploading the changeset.
* I would expect that the DWG would redact the potential name issues from
all the prior versions.
* Next a MapRoulette challenge can be constructed with the
chdr_USA_AZ_name_fixup_required search target.  The challenge will be
localized to Arizona because of the name change.
* The Tiger 2015 layer will be a good source to use name correction in the
challenge.  Others without the Tiger name source will not be so lucky.

The questions and observations:
* chdr is the last modifier of 2980 out of 5696 ways, 52%. These ways would
appear like the first and last introduction of the potential damage.
* The highest way version on one example way was 24.  Does this mean that
the redaction would have to be performed in all the 24 revisions?
* I came in at 1685 out of 5969 ways, 30%.  Even though I can rationalize
and observe that chdr's street name matches the Tiger 2015 layer, I cannot
prove or disprove this. I'd rather make a proactive resolution to the issue
than wait for additional edits on these ways.
* It took a long time for the name change to complete in JOSM once the last
remote control command was issued.

DWG you are up. Please redact the name issue in the prior versions of the
ways in Change set # 51502902.

Once the DWG is finished, it would be great if Martijn could assist me
develop a MapRoulette challenge.

Regards,
Greg
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[joost schouppe]

2017-08-28 10:27 GMT+02:00 Simon Poole :

> What surprises me most about the discussion up to now that it is
> centered around the impact on the US and Canada were the removals are
> rather small both relatively and absolute*, actually are more at the
> nuisance level than anything else, and can easily be added back, likely
> in a couple of days  from sources that are already pre-approved as
> reference material.
>
> Our concern should be more about Mexico, Brazil and other countries
> where it is at least not obvious to me if the local communities are
> aware of the issue and if we have any plan at all how we possibly could
> mitigate the impact.
>

I can't say I'm surprised by that myself. This is a conversation in
English, on a mailing list, bot things not very popular in Latin America.
I've posted this thread to the Latam telegram group, with an offer to
translate if needed.

-- 
Joost Schouppe
OpenStreetMap  |
Twitter  | LinkedIn
 | Meetup

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Frederik Ramm]

Hi,

On 08/28/2017 10:27 AM, Simon Poole wrote:
> Our concern should be more about Mexico, Brazil and other countries
> where it is at least not obvious to me if the local communities are
> aware of the issue and if we have any plan at all how we possibly could
> mitigate the impact. I know that Brazil has already had multiple such
> issues in the past, is Wille Marcel on this list?

At least for São Paulo, DWG has been told by a Brazilian user:

> the city of São Paulo has made available all the names of highway, buildings, 
>  and other information in the site 
> http://geosampa.prefeitura.sp.gov.br/PaginasPublicas/_SBC.aspx
> Therefore, any edition of streets that he has done in the city of São Paulo 
> can be recovered with official data.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Christoph Hormann]

On Monday 28 August 2017, Paul Norman wrote:
>
> As the publisher of the OSM database, the OSMF has various legal
> obligations. When we become aware of data that has been illegally
> copied into OSM we need to stop distributing that data, generally by
> deleting it and redacting the old versions so they are no longer
> accessible. It's worth discussing if we can refine the identification
> of data illegally copied data, but we need to remove it in the end,
> regardless of if we want to.

+1.

In fact reading this thread i cannot really believe this discussion is 
actually happening, that there seem to be people who think that a large 
scale unauthorized use of data can be 'healed' by rubber stamping it 
with a 'review' of the names.

If you think that redaction is pointless, unnecessary, inappropriate or 
similar the right thing to do is to lobby your legislative body to 
change database rights legislation, not to just ignore the law (which i 
am sure none of you would propose to do if you were personally liable).

As has been mentioned by others replacing the problematic data with 
information from other sources (either on-the-ground knowledge or open 
data) is something that can be done both before and after a redaction 
without problems.  And Frederik already indicated the DWG supports 
these efforts.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Simon Poole]

What surprises me most about the discussion up to now that it is
centered around the impact on the US and Canada were the removals are
rather small both relatively and absolute*, actually are more at the
nuisance level than anything else, and can easily be added back, likely
in a couple of days  from sources that are already pre-approved as
reference material.

Our concern should be more about Mexico, Brazil and other countries
where it is at least not obvious to me if the local communities are
aware of the issue and if we have any plan at all how we possibly could
mitigate the impact. I know that Brazil has already had multiple such
issues in the past, is Wille Marcel on this list?

Simon

* the redaction a couple of weeks back in Switzerland removed roughly
10% of the names from "residential" streets to but the US and Canadian
numbers in to perspective and there is currently no usable other source
for the names, and many of them are not surveable.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Warin]

The guidelines were formulated for data additions. This should guard 
against things being entered into the data base that are questionable.


This is a data deletion - a very different thing. In this case it is 
required, ethically at least.


I have looks at some 6 in 'my area' and they all should be removed.
That is some 6 out of ~2,600... don't have time for more at the moment. 
A poor sample size but 100% for deletion.
Again for 'my area' there is no easy copyright free method of name 
verification - they will all have to go.


On 28-Aug-17 10:56 AM, john whelan wrote:
>I haven't seen any compelling evidence or discussion about whether or 
not the data in question was illegally copied into OSM. All we have to 
go on is the first paragraph of Frederik's initial post. I'm not 
questioning Frederik's (or any DWG members') passion or dedication to 
the project, but we are the contributors and I would hope we can 
expect some extra modicum of transparency when a proposal of this 
magnitude is made.


It has been brought to the OSMF's attention.  It has been verified 
that there are "Easter Eggs" from Google are in there.   I must say 
that I agree with Paul Norman's point of view, in this case there is 
no choice.


Having said that there are costs involved in cleaning it up even if 
its only people time.


The decision to me lies between deleting the value in name="xyz 
street" for all the highways touched or seeing if we reduce the work 
by being more selective and verifying some of the names.


Unfortunately if we want to ask someone to remove data copied from OSM 
in the future our case is much stronger if we have deleted all the 
suspect data ourselves on this occasion when it has been brought to 
our attention and verified that there are "Easter eggs" in our data 
and I think you have to take that into account.


Cheerio John



On 27 August 2017 at 20:40, Ian Dees > wrote:


On Sun, Aug 27, 2017 at 7:20 PM, Paul Norman > wrote:

On 8/27/2017 10:29 AM, Ian Dees wrote:


I strongly disagree. As a group of people who have
received extra-judicial powers in the OSM community, they
should be expected to follow community guidelines to a
higher degree than the rest of the community.


As the publisher of the OSM database, the OSMF has various
legal obligations. When we become aware of data that has been
illegally copied into OSM we need to stop distributing that
data, generally by deleting it and redacting the old versions
so they are no longer accessible. It's worth discussing if we
can refine the identification of data illegally copied data,
but we need to remove it in the end, regardless of if we want to.


I haven't seen any compelling evidence or discussion about whether
or not the data in question was illegally copied into OSM. All we
have to go on is the first paragraph of Frederik's initial post.
I'm not questioning Frederik's (or any DWG members') passion or
dedication to the project, but we are the contributors and I would
hope we can expect some extra modicum of transparency when a
proposal of this magnitude is made.

I'm glad this discussion is happening now, but I hope we can
expect to see it happen again if something else comes up in the
future.

___
talk mailing list
talk@openstreetmap.org 
https://lists.openstreetmap.org/listinfo/talk





___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[john whelan]

>I haven't seen any compelling evidence or discussion about whether or not
the data in question was illegally copied into OSM. All we have to go on is
the first paragraph of Frederik's initial post. I'm not questioning
Frederik's (or any DWG members') passion or dedication to the project, but
we are the contributors and I would hope we can expect some extra modicum
of transparency when a proposal of this magnitude is made.

It has been brought to the OSMF's attention.  It has been verified that
there are "Easter Eggs" from Google are in there.   I must say that I agree
with Paul Norman's point of view, in this case there is no choice.

Having said that there are costs involved in cleaning it up even if its
only people time.

The decision to me lies between deleting the value in name="xyz street" for
all the highways touched or seeing if we reduce the work by being more
selective and verifying some of the names.

Unfortunately if we want to ask someone to remove data copied from OSM in
the future our case is much stronger if we have deleted all the suspect
data ourselves on this occasion when it has been brought to our attention
and verified that there are "Easter eggs" in our data and I think you have
to take that into account.

Cheerio John



On 27 August 2017 at 20:40, Ian Dees  wrote:

> On Sun, Aug 27, 2017 at 7:20 PM, Paul Norman  wrote:
>
>> On 8/27/2017 10:29 AM, Ian Dees wrote:
>>
>>>
>>> I strongly disagree. As a group of people who have received
>>> extra-judicial powers in the OSM community, they should be expected to
>>> follow community guidelines to a higher degree than the rest of the
>>> community.
>>>
>>
>> As the publisher of the OSM database, the OSMF has various legal
>> obligations. When we become aware of data that has been illegally copied
>> into OSM we need to stop distributing that data, generally by deleting it
>> and redacting the old versions so they are no longer accessible. It's worth
>> discussing if we can refine the identification of data illegally copied
>> data, but we need to remove it in the end, regardless of if we want to.
>
>
> I haven't seen any compelling evidence or discussion about whether or not
> the data in question was illegally copied into OSM. All we have to go on is
> the first paragraph of Frederik's initial post. I'm not questioning
> Frederik's (or any DWG members') passion or dedication to the project, but
> we are the contributors and I would hope we can expect some extra modicum
> of transparency when a proposal of this magnitude is made.
>
> I'm glad this discussion is happening now, but I hope we can expect to see
> it happen again if something else comes up in the future.
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Ian Dees]

On Sun, Aug 27, 2017 at 7:20 PM, Paul Norman  wrote:

> On 8/27/2017 10:29 AM, Ian Dees wrote:
>
>>
>> I strongly disagree. As a group of people who have received
>> extra-judicial powers in the OSM community, they should be expected to
>> follow community guidelines to a higher degree than the rest of the
>> community.
>>
>
> As the publisher of the OSM database, the OSMF has various legal
> obligations. When we become aware of data that has been illegally copied
> into OSM we need to stop distributing that data, generally by deleting it
> and redacting the old versions so they are no longer accessible. It's worth
> discussing if we can refine the identification of data illegally copied
> data, but we need to remove it in the end, regardless of if we want to.


I haven't seen any compelling evidence or discussion about whether or not
the data in question was illegally copied into OSM. All we have to go on is
the first paragraph of Frederik's initial post. I'm not questioning
Frederik's (or any DWG members') passion or dedication to the project, but
we are the contributors and I would hope we can expect some extra modicum
of transparency when a proposal of this magnitude is made.

I'm glad this discussion is happening now, but I hope we can expect to see
it happen again if something else comes up in the future.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Paul Norman]

On 8/27/2017 10:29 AM, Ian Dees wrote:


I strongly disagree. As a group of people who have received 
extra-judicial powers in the OSM community, they should be expected to 
follow community guidelines to a higher degree than the rest of the 
community.


As the publisher of the OSM database, the OSMF has various legal 
obligations. When we become aware of data that has been illegally copied 
into OSM we need to stop distributing that data, generally by deleting 
it and redacting the old versions so they are no longer accessible. It's 
worth discussing if we can refine the identification of data illegally 
copied data, but we need to remove it in the end, regardless of if we 
want to.


Paul Norman
For the OSMF Data Working Group

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Pierre Béland]

John,
Once the way names are redacted, I will revise for Québec following the naming 
rules different from the english part of Canada. ToDo / JOSM should help for 
this.
regard 
 
Pierre 

john whelan wrote :


I suspect Jamie could wave a magic wand for Quebec.
   ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[James]

The odbl=clean could be a way to exclude road names from the removal,
though sources from the info should be given, whether Canvec, Mapillary,
OpenStreetCam or local survey

On Aug 27, 2017 3:29 PM, "john whelan"  wrote:

> There are a couple of issues here.  The first are our users, we don't
> normally think about them but deleting the names at the wrong point in
> OSMAND's cycle could mean missing street names for a period of time.
>
> Second is the problem of some data might be incorrect as a result of the
> source deliberately using invalid names.  The very clean way is to delete
> then retag.
>
> Verifying with a maproulette challenge would work well if we could trust
> all the mappers not to just tick the box either deliberately or by mistake.
>
> Since we have the location of the streets and we have other sources with a
> valid name which would probably vary by country could someone join the two
> together and verify the name in an automated way?  Leaving a much smaller
> list of street names to be verified manually?  I suspect Jamie could wave a
> magic wand for Quebec.
>
> I'm not saying we should do one thing or another here.  I'm attempting to
> analyse the problem and find a solution that impacts as few people as
> possible but gives us clean accurate data at the end of the process.
>
> If we go the verifying route could we take a page out of HOT's process and
> have someone verify them someone validate?
>
> Cheerio John
>
> On 27 Aug 2017 3:04 pm, "Nicolás Alvarez" 
> wrote:
>
>> I don't understand what people mean with 'verifying' objects. We're
>> not trying to find factually-incorrect data. The data is legally
>> tainted. It's questionable whether looking at the current names
>> imported from GMaps, comparing to another source, seeing they match
>> and marking them as "verified" will legally change anything. And it's
>> impossible to know if people are really verifying anything or just
>> blindly marking them as verified.
>>
>> I think the only clean way to solve this is to redact and then re-map
>> from legal sources.
>>
>> --
>> Nicolás
>>
>> 2017-08-27 14:39 GMT-03:00 Frederik Ramm :
>> > Steve:
>> >
>> > thank you for your work. I'll save your list. It appears that others
>> > might be eager to do the same, maybe we can find a good workflow for
>> > that. I wasn't expecting the community to start working on this
>> > pre-redaction but if people prefer that to fixing issues later, it is of
>> > course an option. I certainly prefer out-of-band "marking" of verified
>> > objects to adding a new tag to each!
>> >
>> > Tod:
>> >
>> > On 08/27/2017 07:31 PM, Tod Fitch wrote:
>> >> When you reviewed Orange County, how did you do it so quickly? The
>> only way I know to go through this is looking at each one, one at a time.
>> >
>> > I could of course make a page with links to the ways, even per county if
>> > that helps, or we could upload the list to some suitable tool. Ian
>> > mentioned MapRoulette but I'm not sure if that would make things easier.
>> > I'm certainly happy to try. Maybe Martijn would like to chip in about
>> > MapRoulette?
>> >
>> > Bye
>> > Frederik
>> >
>> > --
>> > Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09"
>> E008°23'33"
>> >
>> > ___
>> > talk mailing list
>> > talk@openstreetmap.org
>> > https://lists.openstreetmap.org/listinfo/talk
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Greg Morgan]

On Sun, Aug 27, 2017 at 12:01 PM, Nicolás Alvarez  wrote:

> I don't understand what people mean with 'verifying' objects. We're
> not trying to find factually-incorrect data. The data is legally
> tainted. It's questionable whether looking at the current names
> imported from GMaps, comparing to another source, seeing they match
> and marking them as "verified" will legally change anything.



> And it's
> impossible to know if people are really verifying anything or just
> blindly marking them as verified.
>
>
Nicolás is there any chance you can refrain from slimming the community
like that?



> I think the only clean way to solve this is to redact and then re-map
> from legal sources.
>

If you are in another country than the US and Canada, then you may not have
a second legal source and you would make this statement.  In my case, I
have had a series of tiger name layers work from over the years..  So what
Fredrick wants to do based on his list is to wipe out my work with his
purposed blind revert.  As an example, here is way
https://www.openstreetmap.org/way/103141172 .  I cleaned up the geometry
two years ago.  I added access tags and what not.  I make it a habit of
removing all tiger tags when I am finished cleaning up and verifying names
with the tiger layer.  Way 103141172 is on Fredrick's list.  If he performs
is revert, then I am going to have to go back and add the name back that I
have already checked on.  Hence, I do not agree with either of your
statements.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[john whelan]

There are a couple of issues here.  The first are our users, we don't
normally think about them but deleting the names at the wrong point in
OSMAND's cycle could mean missing street names for a period of time.

Second is the problem of some data might be incorrect as a result of the
source deliberately using invalid names.  The very clean way is to delete
then retag.

Verifying with a maproulette challenge would work well if we could trust
all the mappers not to just tick the box either deliberately or by mistake.

Since we have the location of the streets and we have other sources with a
valid name which would probably vary by country could someone join the two
together and verify the name in an automated way?  Leaving a much smaller
list of street names to be verified manually?  I suspect Jamie could wave a
magic wand for Quebec.

I'm not saying we should do one thing or another here.  I'm attempting to
analyse the problem and find a solution that impacts as few people as
possible but gives us clean accurate data at the end of the process.

If we go the verifying route could we take a page out of HOT's process and
have someone verify them someone validate?

Cheerio John

On 27 Aug 2017 3:04 pm, "Nicolás Alvarez"  wrote:

> I don't understand what people mean with 'verifying' objects. We're
> not trying to find factually-incorrect data. The data is legally
> tainted. It's questionable whether looking at the current names
> imported from GMaps, comparing to another source, seeing they match
> and marking them as "verified" will legally change anything. And it's
> impossible to know if people are really verifying anything or just
> blindly marking them as verified.
>
> I think the only clean way to solve this is to redact and then re-map
> from legal sources.
>
> --
> Nicolás
>
> 2017-08-27 14:39 GMT-03:00 Frederik Ramm :
> > Steve:
> >
> > thank you for your work. I'll save your list. It appears that others
> > might be eager to do the same, maybe we can find a good workflow for
> > that. I wasn't expecting the community to start working on this
> > pre-redaction but if people prefer that to fixing issues later, it is of
> > course an option. I certainly prefer out-of-band "marking" of verified
> > objects to adding a new tag to each!
> >
> > Tod:
> >
> > On 08/27/2017 07:31 PM, Tod Fitch wrote:
> >> When you reviewed Orange County, how did you do it so quickly? The only
> way I know to go through this is looking at each one, one at a time.
> >
> > I could of course make a page with links to the ways, even per county if
> > that helps, or we could upload the list to some suitable tool. Ian
> > mentioned MapRoulette but I'm not sure if that would make things easier.
> > I'm certainly happy to try. Maybe Martijn would like to chip in about
> > MapRoulette?
> >
> > Bye
> > Frederik
> >
> > --
> > Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
> >
> > ___
> > talk mailing list
> > talk@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Andrew Hain]

There was a script during the licence change that assessed whether or not 
changes to names were copyrightable.

--
Andrew

From: Mikel Maron <mikel.ma...@gmail.com>
Sent: 27 August 2017 19:51:05
To: Greg Morgan; Martijn van Exel
Cc: Talk Openstreetmap; Steve Friedl; Tod Fitch; talk...@openstreetmap.org
Subject: Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by 
user chdr

> we can find a good workflow for that. I wasn't expecting the community to 
> start working on this pre-redaction but if people prefer that to fixing 
> issues later...

Absolutely, let's do this!

Also, Frederik, I think your script picked up false positives. Spot checked in 
DC, and these are expansions of both the street and the quadrant ("St NW" -> 
"Street Northwest"(. Can we fix the script and regen the list?

http://www.openstreetmap.org/way/109419946/history
http://www.openstreetmap.org/way/109431926/history
http://www.openstreetmap.org/way/109431927/history

-Mikel


* Mikel Maron * +14152835207 @mikel s:mikelmaron


On Sunday, August 27, 2017 2:45 PM, Greg Morgan <dr.kludge...@gmail.com> wrote:




On Sun, Aug 27, 2017 at 10:55 AM, Martijn van Exel 
<mart...@openstreetmap.us<mailto:mart...@openstreetmap.us>> wrote:
Happy to help. All we'd need for MapRoulette is a list of locations and a 
proper description of the work we'd expect people to do. Anyone can create the 
challenge but I'd be happy to do it.
Martijn

Martijn,

I'd would be great if you can break this down to an area.  For example, I have 
a list of Arizona streets.  I'd prefer to work on this as an Arizona challenge 
verses one big chdr challenge.

Please Advise,
Greg

___
talk mailing list
talk@openstreetmap.org<mailto:talk@openstreetmap.org>

https://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Andrew Hain]

Presumably it means something like the 
https://wiki.openstreetmap.org/wiki/Tag:odbl%3Dclean tag. It may not be as much 
use hete though.
--
Andrew

From: Nicolás Alvarez <nicolas.alva...@gmail.com>
Sent: 27 August 2017 20:01:16
To: Frederik Ramm
Cc: Talk Openstreetmap; talk...@openstreetmap.org
Subject: Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by 
user chdr

I don't understand what people mean with 'verifying' objects. We're
not trying to find factually-incorrect data. The data is legally
tainted. It's questionable whether looking at the current names
imported from GMaps, comparing to another source, seeing they match
and marking them as "verified" will legally change anything. And it's
impossible to know if people are really verifying anything or just
blindly marking them as verified.

I think the only clean way to solve this is to redact and then re-map
from legal sources.

--
Nicolás

2017-08-27 14:39 GMT-03:00 Frederik Ramm <frede...@remote.org>:
> Steve:
>
> thank you for your work. I'll save your list. It appears that others
> might be eager to do the same, maybe we can find a good workflow for
> that. I wasn't expecting the community to start working on this
> pre-redaction but if people prefer that to fixing issues later, it is of
> course an option. I certainly prefer out-of-band "marking" of verified
> objects to adding a new tag to each!
>
> Tod:
>
> On 08/27/2017 07:31 PM, Tod Fitch wrote:
>> When you reviewed Orange County, how did you do it so quickly? The only way 
>> I know to go through this is looking at each one, one at a time.
>
> I could of course make a page with links to the ways, even per county if
> that helps, or we could upload the list to some suitable tool. Ian
> mentioned MapRoulette but I'm not sure if that would make things easier.
> I'm certainly happy to try. Maybe Martijn would like to chip in about
> MapRoulette?
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Nicolás Alvarez]

I don't understand what people mean with 'verifying' objects. We're
not trying to find factually-incorrect data. The data is legally
tainted. It's questionable whether looking at the current names
imported from GMaps, comparing to another source, seeing they match
and marking them as "verified" will legally change anything. And it's
impossible to know if people are really verifying anything or just
blindly marking them as verified.

I think the only clean way to solve this is to redact and then re-map
from legal sources.

-- 
Nicolás

2017-08-27 14:39 GMT-03:00 Frederik Ramm :
> Steve:
>
> thank you for your work. I'll save your list. It appears that others
> might be eager to do the same, maybe we can find a good workflow for
> that. I wasn't expecting the community to start working on this
> pre-redaction but if people prefer that to fixing issues later, it is of
> course an option. I certainly prefer out-of-band "marking" of verified
> objects to adding a new tag to each!
>
> Tod:
>
> On 08/27/2017 07:31 PM, Tod Fitch wrote:
>> When you reviewed Orange County, how did you do it so quickly? The only way 
>> I know to go through this is looking at each one, one at a time.
>
> I could of course make a page with links to the ways, even per county if
> that helps, or we could upload the list to some suitable tool. Ian
> mentioned MapRoulette but I'm not sure if that would make things easier.
> I'm certainly happy to try. Maybe Martijn would like to chip in about
> MapRoulette?
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Mikel Maron]

> we can find a good workflow for that. I wasn't expecting the community to 
>start working on this pre-redaction but if people prefer that to fixing issues 
>later...
Absolutely, let's do this!
Also, Frederik, I think your script picked up false positives. Spot checked in 
DC, and these are expansions of both the street and the quadrant ("St NW" -> 
"Street Northwest"(. Can we fix the script and regen the list?
http://www.openstreetmap.org/way/109419946/historyhttp://www.openstreetmap.org/way/109431926/history
http://www.openstreetmap.org/way/109431927/history

-Mikel
 * Mikel Maron * +14152835207 @mikel s:mikelmaron 

On Sunday, August 27, 2017 2:45 PM, Greg Morgan  
wrote:
 

 

On Sun, Aug 27, 2017 at 10:55 AM, Martijn van Exel  
wrote:

Happy to help. All we'd need for MapRoulette is a list of locations and a 
proper description of the work we'd expect people to do. Anyone can create the 
challenge but I'd be happy to do it.Martijn

Martijn,
I'd would be great if you can break this down to an area.  For example, I have 
a list of Arizona streets.  I'd prefer to work on this as an Arizona challenge 
verses one big chdr challenge.
Please Advise,Greg
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


   ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Greg Morgan]

On Sun, Aug 27, 2017 at 10:55 AM, Martijn van Exel  wrote:

> Happy to help. All we'd need for MapRoulette is a list of locations and a
> proper description of the work we'd expect people to do. Anyone can create
> the challenge but I'd be happy to do it.
>
> Martijn
>

Martijn,

I'd would be great if you can break this down to an area.  For example, I
have a list of Arizona streets.  I'd prefer to work on this as an Arizona
challenge verses one big chdr challenge.

Please Advise,
Greg
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Greg Morgan]

On Sun, Aug 27, 2017 at 10:29 AM, Ian Dees  wrote:

>
>
> On Aug 27, 2017 11:58, "Yves"  wrote:
>
>  a écrit :
>
>
> Frederik,
>
>
> Thanks for notifying us about this. I hope that you treat this as an
> import or automated edit and follow the rules you would expect to see the
> rest of the community follow
>
>
> Ian,
> To lessen the burden of the DWG, I would say that this thread is
> sufficient to document the redaction, plus whatever documentation the DWG
> usually make.
>
>
> I strongly disagree. As a group of people who have received extra-judicial
> powers in the OSM community, they should be expected to follow community
> guidelines to a higher degree than the rest of the community.
>

I agree with Ian.  Yves what about the DWG lessening the burden on mappers?
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Pierre Béland]

Frederik just answered Steve, but no message was received from Steve on the 
talk list. Bad documentation of the thread and difficulty to follow discussions 
coming from two lists, talk and talk-us. I then suggest that this thread be 
only on talk.

>From the list that Frederik provided earlier (ie. 
>http://www.remote.org/frederik/tmp/chdr.details),it is easy to extract the 
>id's for one region or country and query the Overpass-API as the example below 
>with a list of way's id.

way(id:102501108, 103339404, 103339466, ...);
out meta;>; out meta; 
 
Pierre 


  De : Frederik Ramm 
 À : Tod Fitch ; Steve Friedl  
Cc : Talk Openstreetmap ; talk...@openstreetmap.org
  ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Clifford Snow]

On Sun, Aug 27, 2017 at 6:49 AM, Frederik Ramm  wrote:

>
> Here's a list of way IDs affected, with country and state:
>
> http://www.remote.org/frederik/tmp/chdr.details
>
>
Frederik,
I looked a small sample of the list. For example, way 10012342 [1] in Texas
was only touched by chdr. The name originated with the TIGER import by Dave
Hansen. It doesn't appear to me that at this point a mass deletion, at
least in the US, is appropriate.

As Ian pointed out, using a tool like Maproulette to validate names would
be a more appropriate recourse to insure we chdr did not add copyright
information to OSM. Especially four years later. I believe the US community
can manage to review 18,000 names to insure integrity of our project.

[1] http://www.openstreetmap.org/way/10012342/history

Best,
Clifford
-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Frederik Ramm]

Steve:

thank you for your work. I'll save your list. It appears that others
might be eager to do the same, maybe we can find a good workflow for
that. I wasn't expecting the community to start working on this
pre-redaction but if people prefer that to fixing issues later, it is of
course an option. I certainly prefer out-of-band "marking" of verified
objects to adding a new tag to each!

Tod:

On 08/27/2017 07:31 PM, Tod Fitch wrote:
> When you reviewed Orange County, how did you do it so quickly? The only way I 
> know to go through this is looking at each one, one at a time.

I could of course make a page with links to the ways, even per county if
that helps, or we could upload the list to some suitable tool. Ian
mentioned MapRoulette but I'm not sure if that would make things easier.
I'm certainly happy to try. Maybe Martijn would like to chip in about
MapRoulette?

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Ian Dees]

On Aug 27, 2017 11:58, "Yves"  wrote:

 a écrit :


Frederik,


Thanks for notifying us about this. I hope that you treat this as an import
or automated edit and follow the rules you would expect to see the rest of
the community follow


Ian,
To lessen the burden of the DWG, I would say that this thread is sufficient
to document the redaction, plus whatever documentation the DWG usually
make.


I strongly disagree. As a group of people who have received extra-judicial
powers in the OSM community, they should be expected to follow community
guidelines to a higher degree than the rest of the community.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Frederik Ramm]

Hi Ian,

On 08/27/2017 06:23 PM, Ian Dees wrote:
> Thanks for notifying us about this. I hope that you treat this as an
> import or automated edit and follow the rules you would expect to see
> the rest of the community follow.

Sorry for not being clear about this. I posted this with my DWG hat on,
and this is not an import or an automated edit but an "official" removal
of incompatible data which would be an everyday thing for DWG if it
weren't for the sheer number of contributions and the fact that they are
so old, hence the discussion ahead.

> Please post samples of your changes,

The change will be removing the name tag on all ways in the file I made
available, or reverting to whatever it was before chdr edited the
object. Other properties would not be edited.

If this was not a misunderstanding, and instead you wanted to use the
opportunity to start a discussion about DWG accountability ("every
revert of an automated edit is also an automated edit and hence needs to
be discussed in advance", etc.) then I would kindly ask you to do that
in separate thread with a suitable topic.

> and thanks for working to get buy-in
> from local community.

Let's call it a heads-up for, rather than a buy-in from, because even if
the local community were against it, I don't see a legally clean way
around removing these names.

> Is your plan to revert changes to the name tag made by chdr or will you
> be completely removing the name tag? Personally, I would prefer to see
> the name tag completely removed so we can more easily come back and
> correct it. It might also be better to load this list you posted into
> maproulette or similar so we can systematically validate the name values
> on the ways.

Streets that would end up not having a name will probably show up in
enough QA tools already, no?

I'll do a count of how many names would revert and how many would be
removed altogether. Maybe it is indeed best to simply ditch them all.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Yves]

 a écrit :

Frederik,


Thanks for notifying us about this. I hope that you treat this as an import or 
automated edit and follow the rules you would expect to see the rest of the 
community follow


Ian, 
To lessen the burden of the DWG, I would say that this thread is sufficient to 
document the redaction, plus whatever documentation the DWG usually make. 
Yves ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[James]

Indeed, with the geometry still remaining it will be easy to create a
maproulette task(s) to repair the damage

+1 for name tag redaction

On Aug 27, 2017 12:26 PM, "Ian Dees"  wrote:

> Frederik,
>
> Thanks for notifying us about this. I hope that you treat this as an
> import or automated edit and follow the rules you would expect to see the
> rest of the community follow. Please post samples of your changes, make a
> wiki page for posterity, and thanks for working to get buy-in from local
> community.
>
> Is your plan to revert changes to the name tag made by chdr or will you be
> completely removing the name tag? Personally, I would prefer to see the
> name tag completely removed so we can more easily come back and correct it.
> It might also be better to load this list you posted into maproulette or
> similar so we can systematically validate the name values on the ways.
>
> -Ian
>
> On Sun, Aug 27, 2017 at 8:49 AM, Frederik Ramm 
> wrote:
>
>> Hi,
>>
>>in 2010 I was privately contacted by another OSM user with the
>> suspicion that user "chdr" might be copying names from Google maps
>> (there were few "easter eggs" in Oman that were only on Google and not
>> in the real world, and they suddenly popped up on OSM). "chdr" was
>> contacted at the time, but continued unfazed. In 2013 another mapper
>> lodged a complaint with DWG about edits by chdr, and I emailed chdr
>> asking him about his sources. At that point chdr stopped mapping. He
>> never replied about his sources though, even when I set an ultimatum (of
>> 31st August 2013) threatening to remove all names he contributed if he
>> can't tell us his source. We do have to assume that all names
>> contributed by chdr are copyright violations.
>>
>> (chdr has added names all around the world, making a harmless survey
>> unlikely.)
>>
>> For various reasons I neglected to act on this, and was only reminded
>> now, 5 years later, when DWG received a complaint from a user in Brazil
>> where chdr has even used "source=google" occasionally. (But as I said,
>> the suspicion is that Google was used throughout.)
>>
>> I have now compiled a list of all street names that were contributed by
>> chdr and are still visible today; we're talking about almost 75,000
>> street names world wide. The most affected countries are:
>>
>>   18023 "United States of America"
>>   16345 "Mexico"
>>   15109 "Brazil"
>>6791 "RSA"
>>2802 "Spain"
>>2614 "Australia"
>>1923 "Argentina"
>>1673 "Nigeria"
>>1569 "India"
>>1441 "Canada"
>> 954 "Malaysia"
>> 744 "Botswana"
>> 717 "Philippines"
>> 619 "Indonesia"
>> 553 "Italy"
>> 414 "Turkey"
>> 290 "Hungary"
>> 284 "Chile"
>> 250 "Kenya"
>> 127 "Saudi Arabia"
>> 107 "Paraguay"
>> 106 "Panama"
>> 100 "Morocco"
>>
>> I've left out those countries with less than 100 affected ways.
>>
>> For the US, I can break it down by state:
>>
>>5696 "Arizona"
>>5116 "Texas"
>>2294 "New York"
>>1164 "District of Columbia"
>> 740 "Iowa"
>> 494 "Colorado"
>> 416 "New Jersey"
>> 339 "Illinois"
>> 268 "Michigan"
>> 239 "Pennsylvania"
>> 181 "Missouri"
>> 147 "Georgia"
>> 129 "New Mexico"
>> 123 "North Carolina"
>> 115 "California"
>> 106 "Virginia"
>>
>> The breakdown for Mexico:
>>
>>7749 "Baja California"
>>2084 "Puebla"
>>1964 "Chihuahua"
>>1539 "Coahuila"
>>1161 "Mexico"
>>1040 "Chiapas"
>> 342 "Tamaulipas"
>> 241 "Sonora"
>> 185 "San Luis Potosi"
>> 129 "New Mexico"
>>
>> and Brazil:
>>
>>   10904 "São Paulo"
>>2605 "Paraná"
>> 945 "Rio de Janeiro"
>> 270 "Rio Grande do Sul"
>> 154 "Goiás"
>>
>> and South Africa:
>>
>>4422 "Gauteng"
>> 750 "KwaZulu-Natal"
>> 600 "Eastern Cape"
>> 439 "Western Cape"
>> 400 "Northern Cape"
>> 179 "Mpumalanga"
>>
>> - each time leaving out a couple others under 100.
>>
>> We believe that only names, not geometries have been taken from other
>> maps so we'll remove and redact the names only. In identifying "names
>> contributed by chdr" I took care to really only pick up the names that
>> were introduced by them, not names that were there before, and also when
>> chdr split a way that had a name I will make sure that the newly created
>> way doesn't count as "named by chdr". Additionally, I have ignored those
>> cases where chdr simply performed a TIGER expansion (St->Street etc) of
>> a name that was there before.
>>
>> My process has two weak points (that I am aware of):
>>
>> 1. It doesn't properly "follow" a chrdr-contributed name through way
>> splits performed by other users; if someone has split a way created by
>> chdr, then the name will remain on the bit that was created by this
>> user. This is somewhat unsatisfying but after having manually checked a
>> random sample I think the problem is small enough to be ignored.
>>
>> 2. It is possible 

Re: [OSM-talk] [Talk-us] Redacting 75, 000 street names contributed by user chdr

[Ian Dees]

Frederik,

Thanks for notifying us about this. I hope that you treat this as an import
or automated edit and follow the rules you would expect to see the rest of
the community follow. Please post samples of your changes, make a wiki page
for posterity, and thanks for working to get buy-in from local community.

Is your plan to revert changes to the name tag made by chdr or will you be
completely removing the name tag? Personally, I would prefer to see the
name tag completely removed so we can more easily come back and correct it.
It might also be better to load this list you posted into maproulette or
similar so we can systematically validate the name values on the ways.

-Ian

On Sun, Aug 27, 2017 at 8:49 AM, Frederik Ramm  wrote:

> Hi,
>
>in 2010 I was privately contacted by another OSM user with the
> suspicion that user "chdr" might be copying names from Google maps
> (there were few "easter eggs" in Oman that were only on Google and not
> in the real world, and they suddenly popped up on OSM). "chdr" was
> contacted at the time, but continued unfazed. In 2013 another mapper
> lodged a complaint with DWG about edits by chdr, and I emailed chdr
> asking him about his sources. At that point chdr stopped mapping. He
> never replied about his sources though, even when I set an ultimatum (of
> 31st August 2013) threatening to remove all names he contributed if he
> can't tell us his source. We do have to assume that all names
> contributed by chdr are copyright violations.
>
> (chdr has added names all around the world, making a harmless survey
> unlikely.)
>
> For various reasons I neglected to act on this, and was only reminded
> now, 5 years later, when DWG received a complaint from a user in Brazil
> where chdr has even used "source=google" occasionally. (But as I said,
> the suspicion is that Google was used throughout.)
>
> I have now compiled a list of all street names that were contributed by
> chdr and are still visible today; we're talking about almost 75,000
> street names world wide. The most affected countries are:
>
>   18023 "United States of America"
>   16345 "Mexico"
>   15109 "Brazil"
>6791 "RSA"
>2802 "Spain"
>2614 "Australia"
>1923 "Argentina"
>1673 "Nigeria"
>1569 "India"
>1441 "Canada"
> 954 "Malaysia"
> 744 "Botswana"
> 717 "Philippines"
> 619 "Indonesia"
> 553 "Italy"
> 414 "Turkey"
> 290 "Hungary"
> 284 "Chile"
> 250 "Kenya"
> 127 "Saudi Arabia"
> 107 "Paraguay"
> 106 "Panama"
> 100 "Morocco"
>
> I've left out those countries with less than 100 affected ways.
>
> For the US, I can break it down by state:
>
>5696 "Arizona"
>5116 "Texas"
>2294 "New York"
>1164 "District of Columbia"
> 740 "Iowa"
> 494 "Colorado"
> 416 "New Jersey"
> 339 "Illinois"
> 268 "Michigan"
> 239 "Pennsylvania"
> 181 "Missouri"
> 147 "Georgia"
> 129 "New Mexico"
> 123 "North Carolina"
> 115 "California"
> 106 "Virginia"
>
> The breakdown for Mexico:
>
>7749 "Baja California"
>2084 "Puebla"
>1964 "Chihuahua"
>1539 "Coahuila"
>1161 "Mexico"
>1040 "Chiapas"
> 342 "Tamaulipas"
> 241 "Sonora"
> 185 "San Luis Potosi"
> 129 "New Mexico"
>
> and Brazil:
>
>   10904 "São Paulo"
>2605 "Paraná"
> 945 "Rio de Janeiro"
> 270 "Rio Grande do Sul"
> 154 "Goiás"
>
> and South Africa:
>
>4422 "Gauteng"
> 750 "KwaZulu-Natal"
> 600 "Eastern Cape"
> 439 "Western Cape"
> 400 "Northern Cape"
> 179 "Mpumalanga"
>
> - each time leaving out a couple others under 100.
>
> We believe that only names, not geometries have been taken from other
> maps so we'll remove and redact the names only. In identifying "names
> contributed by chdr" I took care to really only pick up the names that
> were introduced by them, not names that were there before, and also when
> chdr split a way that had a name I will make sure that the newly created
> way doesn't count as "named by chdr". Additionally, I have ignored those
> cases where chdr simply performed a TIGER expansion (St->Street etc) of
> a name that was there before.
>
> My process has two weak points (that I am aware of):
>
> 1. It doesn't properly "follow" a chrdr-contributed name through way
> splits performed by other users; if someone has split a way created by
> chdr, then the name will remain on the bit that was created by this
> user. This is somewhat unsatisfying but after having manually checked a
> random sample I think the problem is small enough to be ignored.
>
> 2. It is possible that, like with a recent case in Switzerland where I
> had to do a similar redaction, some of these chdr-contributed names will
> have been confirmed by others in a survey, i.e. someone else surveyed
> the area and checked the name, but saw no need to change it in any way
> since it was already correct. Sadly my process will now remove the name
> even though,