Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

On Thu, Apr 27, 2017 at 05:54:39PM +0200, Jochen Topf wrote:
> Date: Thu, 27 Apr 2017 17:54:39 +0200
> From: Jochen Topf <joc...@remote.org>
> To: hot@openstreetmap.org
> Subject: Re: [HOT] Broken (multi)polygon cleanup
> 
> On Thu, Apr 27, 2017 at 03:36:11PM +0200, Jochen Topf wrote:
> > Because there is a large number of those in HOT areas, I have split
> > those off into their own challenge
> > (http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).
> 
> Sorry, that was the wrong URL. The correct one is:
> http://maproulette.org/map/2499

Sorry again. I pulled the challenge and recreated a new one without the
cases where the "spike" had zero-length. That was understandably confusing.

New link is at http://maproulette.org/map/2500.

Info is still at http://area.jochentopf.com/fixing.html#spikes-buildings .

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

On Thu, Apr 27, 2017 at 03:36:11PM +0200, Jochen Topf wrote:
> Because there is a large number of those in HOT areas, I have split
> those off into their own challenge
> (http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).

Sorry, that was the wrong URL. The correct one is:
http://maproulette.org/map/2499

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Michael Heißmeier]

Hi Donal,

Donal Hunt, 2017-04-27 16:26:
Re: http://area.jochentopf.com/fixing.html#spikes-buildings 

I looked at one of the issues flagged (e.g. 
http://maproulette.org/map/2498/2186222) and don't really understand what the 
"spike" problem is. Are there examples of how to identify the issue? Are these 
issues only fixable in JOSM or can one use ID?




According to Jochen's description on the webpage a spike is a line which returns 
in itself. In the case you are asking for this line has zero length. There are 
three nodes in the way for this house which are at the same position. This can 
be visualized in JOSM using its built-in validator which detects duplicate nodes:



The house in the upper left of these nodes has the additional problem that it 
contains one of the twice, hence the zero-length spike:




I don't think that you can detect something like this in iD. But you can clearly 
see the ones with non-vanishing spikes (like 
http://maproulette.org/map/2499/2187301) in iD.


If you use iD my suggestion is to skip those without visible spikes and leave 
them to JOSM users.


Best Regards


--
/Michael
(osm:michael63) /
___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

Hi!

I just posted a new Maproulette challenge for fixing one particular case
of broken polygons: Building ways with "spikes". Details are at
http://area.jochentopf.com/fixing.html#spikes-buildings .

Because there is a large number of those in HOT areas, I have split
those off into their own challenge
(http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).
Would be great to get some help fixing those from the HOT community.

Jochen

On Wed, Mar 15, 2017 at 04:06:05PM +0100, Jochen Topf wrote:
> Date: Wed, 15 Mar 2017 16:06:05 +0100
> From: Jochen Topf <joc...@remote.org>
> To: hot@openstreetmap.org
> Subject: [HOT] Broken (multi)polygon cleanup
> 
> Hi!
> 
> There are many broken (multi)polygon geometries in OSM data leading to
> all sorts of problems. I have started an effort to get this cleaned up.
> You can read more about this at http://area.jochentopf.com/ .
> 
> This effort has been running for about a month now and we can already see
> some good results. See 
> https://github.com/osmlab/fixing-polygons-in-osm/issues/15
> for some news about what's been happening.
> 
> I am writing to the HOT list today, because a not-so-small part of the
> problems come from HOT mappers. For instance, looking at one particular
> problem here (spikes in closed ways), I am seeing about 5800 errors, of
> which about 1600 are in areas covered by the HOT tasking manager. (Of
> couse not all of these have been done in the course of HOT mapping, but
> probably most of them.)
> 
> I am reaching out to some of the regional OSM communities and also to
> HOT, because I want to a) ask for the help of all the communities in OSM
> to fix those errors and b) generally raise awareness, so those problems
> can be minimized in the future.
> 
> Please go to the web sites linked above and read up on what the problems
> are and how we are fixing them. I also want to offer my help in
> preparing special data extracts or Maproulette challenges etc. to help
> the HOT community participate in the cleaning up effort in a way that
> makes sense to you. I could, for instance, put those 1600 errors
> mentioned above in a special Maproulette challenge, if that would be a
> good way for HOT mappers to work on them.
> 
> Tell me what you think. I appreciate any feedback.
> 
> Jochen
> -- 
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688
> 
> ___
> HOT mailing list
> HOT@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/hot

-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[john whelan]

I think you mean tile but the idea has merit.

Thanks John

On 18 March 2017 at 12:43, Coast Limit  wrote:

> I know this discussion is focused on cleaning up these issues, but — as a
> relatively new mapper — can I make a suggestion on how we might better
> limit mistakes in the first place?
>
> John Whelan said earlier, "What I have noticed is one mapper will misstag
> then others will follow.” I think this is true, and likely goes beyond
> tagging — I suspect new mappers are looking for frames of reference to
> judge their own work by, and end up just mimicking one another’s poor
> mapping methods. This is one way we might be getting “experienced” mappers
> (i.e., thousands of edits) who are still adding a lot of problems to the
> map.
>
> My suggestion:
>
> It would be nice if HOT projects started with a single task already mapped
> and completed by experienced mappers, and approved by the project manager,
> as a “reference task” for the project. Highlight it with a unique color.
>
> That way new mappers have an authoritative reference point for that task —
> they can see which imagery it should be aligned with, the level of road
> detail expected, the different types of tags attached to areas, which
> people they might message with questions, etc.
>
> It would give motivated new mappers something trustworthy to mimic.
>
> Thanks,
>
> CoastLimit
>
>
> ___
> HOT mailing list
> HOT@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/hot
>
___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Michael Heißmeier]

Hi all,

Jochen Topf, 2017-03-18 11:20:

On Fri, Mar 17, 2017 at 10:48:00AM -0400, john whelan wrote:

I've read it through but remember like many mappers I do not have a Ph.D.
in GIS, just a degree in Chemistry so I'm usually not too bad on logic.

The first thing that stands out is the message is too complex for the
intended audience.

Yes, and no. The documentation we have now is trying to be complete and
thorough. The intended audience are, first, power mappers. I am totally
aware that it is too much for the average mapper, but that's why I need
everybodies help not only the fix things, but also to translate the
problem into words that the different audiences can understand.

I know that HOT has capable mappers that understand OSM well and have
written documentation for new HOT mappers to use. I hope I can reach
those people here so they can help.


The second "The *new style* tagging option is the recommended tagging
option these days, but some mappers still disagree."  So we don't have
complete agreement, fine.

Well, that was me being overcautious. I haven't heard any disagreements
in the last months. So I don't think there is any major disagreements
left. I have removed that sentence from the documentation.


The third is the main problem areas seem to be some rendering systems
prefer one method over the other.  "Are we mapping for a specific rendering
system now?"

No, we are not. I am not sure why you got that impression from the
documentation. Can you point me to a specific sentence? I have
highlighted Osm2pgsql, because it is by far the most often used basis
for rendering maps and it is what powers the main OSM map. So changes
there will affect more people. And it also means that basically every
other system is trying to mimic what Osm2pgsql is doing (right or
wrong), because they have to keep "in sync" with the main map that OSM
mappers use to verify their work.

Currently all rendering systems have to handle old- and new-style
multipolygons and in the future they wont have to do that any more. This
makes all systems simpler.


I suspect if we concentrate on unclosed ways and duplicate segments then
this is something that can be done without a Ph.D. in GIS concepts and
there is no disagreement.  Leave the more complex problem solving to others.

I totally agree. That's why I have been splitting up the work into small
Maproulette tasks, each with a very easy to understand and concrete
description of what the fixing task is. I don't expect most mappers to
be able to solve complex multipolygon cases. But there are plenty of
simple cases around.


I gave the Maproulette tasks a try yesterday evening and found them easy to 
tackle. The advantage here is that they are typically localized to a small area, 
you do not have to take lots of data into account.


For the bigger picture, OSM Inspector is the tool of choice but I agree that 
many errors reported here are not easy to fix.
I will try to get an overview of errors made by relatively new contributors and 
see how we can improve documentation and learning materials. Errors made are not 
always what you think of when writing documentation, cf. this "single" building 
which contains self-intersections:




Not to forget the biggest multipolygon of all (although not technically 
organized as one), the coastline, which is more often broken than not.





I would recommend if you wish to use the resources of HOT that you talk
nicely to the HOT training group and see if they can sort out the message
and what training needs to be given to support these efforts including what
you would like mappers to avoid when mapping.

I am hoping these people are listening here. (Are you?) Or do I have to
go somewhere else to talk to them?


Yes, people active in the Training Working Group are listening here as well. We 
are aware that new mappers need guidance and made substantial improvements over 
the past months. Out chair, Nick Allen, mentioned this last month in his e-mail 
to the hot list 
(http://lists.openstreetmap.org/pipermail/hot/2017-February/012977.html). We are 
always open to suggestions for further material to be included there.


Best Regards

/Michael
(osm:michael63)
//

/PS: John,

I have yet to see a HOT project that asks mappers to map country outlines.

I agree that these are rare but they do exist 
(http://tasks.hotosm.org/project/1407)


___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

On Fri, Mar 17, 2017 at 10:48:00AM -0400, john whelan wrote:
> I've read it through but remember like many mappers I do not have a Ph.D.
> in GIS, just a degree in Chemistry so I'm usually not too bad on logic.
> 
> The first thing that stands out is the message is too complex for the
> intended audience.

Yes, and no. The documentation we have now is trying to be complete and
thorough. The intended audience are, first, power mappers. I am totally
aware that it is too much for the average mapper, but that's why I need
everybodies help not only the fix things, but also to translate the
problem into words that the different audiences can understand.

I know that HOT has capable mappers that understand OSM well and have
written documentation for new HOT mappers to use. I hope I can reach
those people here so they can help.

> The second "The *new style* tagging option is the recommended tagging
> option these days, but some mappers still disagree."  So we don't have
> complete agreement, fine.

Well, that was me being overcautious. I haven't heard any disagreements
in the last months. So I don't think there is any major disagreements
left. I have removed that sentence from the documentation.

> The third is the main problem areas seem to be some rendering systems
> prefer one method over the other.  "Are we mapping for a specific rendering
> system now?"

No, we are not. I am not sure why you got that impression from the
documentation. Can you point me to a specific sentence? I have
highlighted Osm2pgsql, because it is by far the most often used basis
for rendering maps and it is what powers the main OSM map. So changes
there will affect more people. And it also means that basically every
other system is trying to mimic what Osm2pgsql is doing (right or
wrong), because they have to keep "in sync" with the main map that OSM
mappers use to verify their work.

Currently all rendering systems have to handle old- and new-style
multipolygons and in the future they wont have to do that any more. This
makes all systems simpler.

> I suspect if we concentrate on unclosed ways and duplicate segments then
> this is something that can be done without a Ph.D. in GIS concepts and
> there is no disagreement.  Leave the more complex problem solving to others.

I totally agree. That's why I have been splitting up the work into small
Maproulette tasks, each with a very easy to understand and concrete
description of what the fixing task is. I don't expect most mappers to
be able to solve complex multipolygon cases. But there are plenty of
simple cases around.

> I would recommend if you wish to use the resources of HOT that you talk
> nicely to the HOT training group and see if they can sort out the message
> and what training needs to be given to support these efforts including what
> you would like mappers to avoid when mapping.

I am hoping these people are listening here. (Are you?) Or do I have to
go somewhere else to talk to them?

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Pierre Béland]

Hi Jochen
Your focus on these problems more related to the softwares process is quite 
interesting and valuable. 

While I continue to focus on the Haiti south west peninsula after the Matthew 
hurricane landing in october 2016,  I looked today at multipolygon relations 
misuse.
For Buildings with only outer rings, I merged the ways if necessary, removed 
the relation and added the building tag to the ways.
This Overpass query shows some of these multipolygons before correction.
http://overpass-turbo.eu/s/nBb

I also found and incorrect use is multipolygon relation as a collection of 
buildings, removed the relation and corrected the tagging. There were 7 
buildings in this relation. This does not respect the OSM data model and we 
cannot easily count the number of building in GIS softwares. The mapper who 
added this relation has more then 600 days of contribution and hence classified 
by Pascal Neis as Great Mapper (Very Active). 
http://overpass-turbo.eu/s/nB9

The next problem I will look at is ways with only the area tag. 

Pierre 





   ___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[john whelan]

I've read it through but remember like many mappers I do not have a Ph.D.
in GIS, just a degree in Chemistry so I'm usually not too bad on logic.

The first thing that stands out is the message is too complex for the
intended audience.

The second "The *new style* tagging option is the recommended tagging
option these days, but some mappers still disagree."  So we don't have
complete agreement, fine.

The third is the main problem areas seem to be some rendering systems
prefer one method over the other.  "Are we mapping for a specific rendering
system now?"

I get the impression that the main problem is areas of landuse or defining
countries.  The defining country side I leave to specialists.
Landuse=forest type things are often out of date and problematical.

At a basic level I think we are talking using JOSM to do this work.  I hate
to say it but despite my efforts JOSM is still not main stream for HOT
mapathons.  So essentially we will confuse the audience and confused HOT
mappers may lead to other problems.

I suspect if we concentrate on unclosed ways and duplicate segments then
this is something that can be done without a Ph.D. in GIS concepts and
there is no disagreement.  Leave the more complex problem solving to others.

I have yet to see a HOT project that asks mappers to map country outlines.
I have seen some that ask for landuse=residential but that is about as
complicated as it normally gets.  There might be one or two odd ones that
ask for landuse but these days they are rare.  The bulk of the mapping is
buildings and highways with the occasional river thrown in and I don't
think these are problem areas to you.

There have been some imports in the HOT areas of questionable data quality
some are fine but that is a different matter to HOT mapping.

I would recommend if you wish to use the resources of HOT that you talk
nicely to the HOT training group and see if they can sort out the message
and what training needs to be given to support these efforts including what
you would like mappers to avoid when mapping.

Thanks

Cheerio John



On 17 March 2017 at 09:43, Jochen Topf <joc...@remote.org> wrote:

> Hi!
>
> did you read those documents linked from http://area.jochentopf.com/ ?
>
> https://github.com/osmlab/fixing-polygons-in-osm/blob/
> master/doc/background.md
> https://github.com/osmlab/fixing-polygons-in-osm/blob/
> master/doc/problems.md
> https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/faq.md
>
> If there is something unclear after reading them, please ask, and I am
> happy to explain.
>
> Jochen
>
> On Fri, Mar 17, 2017 at 08:30:27AM -0400, john whelan wrote:
> > Date: Fri, 17 Mar 2017 08:30:27 -0400
> > From: john whelan <jwhelan0...@gmail.com>
> > To: Jochen Topf <joc...@remote.org>
> > Cc: "hot@openstreetmap.org" <hot@openstreetmap.org>, Blake Girardot
> HOT/OSM
> >  <blake.girar...@hotosm.org>
> > Subject: Re: [HOT] Broken (multi)polygon cleanup
> >
> > What isn't clear to me is just what you are trying to fix.  The term
> means
> > nothing to me.  I am correcting duplicate segments and unclosed ways
> which
> > appear to be part of your problem as I work.  Perhaps someone can explain
> > what you are after in simpler terms because if we don't understand the
> > problem then trying to come up with ways to avoid it in the future isn't
> > going to happen.
> >
> > Thanks John
> >
> > On 17 March 2017 at 08:16, Jochen Topf <joc...@remote.org> wrote:
> >
> > > Hi!
> > >
> > > I am sure there are many other problems, but can we please keep to the
> > > issue at hand in this thread? I am not trying to solve all problems, I
> > > am trying to solve one particular set of problem related to
> > > (multi)polygons.
> > >
> > > And I didn't pick that particular problem at random. This is something
> > > that affects everybody. The short of it is: Due to upcoming software
> > > changes, some (multi)polygons that are rendered now will disappear from
> > > the map or look different in the future. Again, you can find all the
> > > details about the problems and of what we are trying to achieve at
> > > http://area.jochentopf.com/ .
> > >
> > > We are having great success working on these problems. You can see the
> > > stats at http://area.jochentopf.com/stats/ . This doesn't mean that
> the
> > > approach will be the best one for other problems, and it doesn't mean
> we
> > > can't do even better than we are doing until now with this problem.
> > >
> > > Btw: I am deliberately talking about "(multi)polygons". This is not
> > > something that on

Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

Hi!

did you read those documents linked from http://area.jochentopf.com/ ?

https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/background.md
https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/problems.md
https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/faq.md

If there is something unclear after reading them, please ask, and I am
happy to explain.

Jochen

On Fri, Mar 17, 2017 at 08:30:27AM -0400, john whelan wrote:
> Date: Fri, 17 Mar 2017 08:30:27 -0400
> From: john whelan <jwhelan0...@gmail.com>
> To: Jochen Topf <joc...@remote.org>
> Cc: "hot@openstreetmap.org" <hot@openstreetmap.org>, Blake Girardot HOT/OSM
>  <blake.girar...@hotosm.org>
> Subject: Re: [HOT] Broken (multi)polygon cleanup
> 
> What isn't clear to me is just what you are trying to fix.  The term means
> nothing to me.  I am correcting duplicate segments and unclosed ways which
> appear to be part of your problem as I work.  Perhaps someone can explain
> what you are after in simpler terms because if we don't understand the
> problem then trying to come up with ways to avoid it in the future isn't
> going to happen.
> 
> Thanks John
> 
> On 17 March 2017 at 08:16, Jochen Topf <joc...@remote.org> wrote:
> 
> > Hi!
> >
> > I am sure there are many other problems, but can we please keep to the
> > issue at hand in this thread? I am not trying to solve all problems, I
> > am trying to solve one particular set of problem related to
> > (multi)polygons.
> >
> > And I didn't pick that particular problem at random. This is something
> > that affects everybody. The short of it is: Due to upcoming software
> > changes, some (multi)polygons that are rendered now will disappear from
> > the map or look different in the future. Again, you can find all the
> > details about the problems and of what we are trying to achieve at
> > http://area.jochentopf.com/ .
> >
> > We are having great success working on these problems. You can see the
> > stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
> > approach will be the best one for other problems, and it doesn't mean we
> > can't do even better than we are doing until now with this problem.
> >
> > Btw: I am deliberately talking about "(multi)polygons". This is not
> > something that only affects multipolygon relations, but all polygons,
> > even the ones created from closed ways. They are simpler than mp
> > relations, but that doesn't mean they can't have problems.
> >
> > Jochen
> >
> > On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:
> > > Date: Thu, 16 Mar 2017 17:23:47 -0400
> > > From: john whelan <jwhelan0...@gmail.com>
> > > To: Jochen Topf <joc...@remote.org>
> > > Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM
> > >  <blake.girar...@hotosm.org>
> > > Subject: Re: [HOT] Broken (multi)polygon cleanup
> > >
> > > This is similar to my pulling in Africa a country at a time and using
> > JOSM
> > > to check for errors.
> > >
> > > What I have noticed is one mapper will misstag then others will follow.
> > So
> > > sanitising the area or checking it for errors before the new HOT mappers
> > > start would probably improve the overall data quality as they tend to
> > > follow what is already mapped.
> > >
> > > Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings
> > in
> > > one small area by the same mapper left as area=yes.  That's quick and
> > easy
> > > to fix.
> > >
> > > Recently I came across 42 highways left untagged by the same mapper in
> > the
> > > same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
> > >
> > > Crossing highways often I'll see a town where the highway residentials
> > are
> > > laid out in a grid but the junctions aren't nodes.
> > >
> > > Another nice recent one was a batch of settlements that had been tagged
> > > highway=unclassified rather than landuse=residential.  Searching for
> > > highways above unclassified and crossing ways picked those out quite
> > > quickly.
> > >
> > > These types of errors are handled more quickly directly in JOSM than
> > > involving maproulette.
> > >
> > > Even the some other errors can be loaded up in the JOSM  to-do plugin.
> > >
> > > I've been cleaning up in Nigeria for a while now.  Making repeated passes
> > > over the same areas over time.  I'm seeing fewer errors s

Re: [HOT] Broken (multi)polygon cleanup

[john whelan]

What isn't clear to me is just what you are trying to fix.  The term means
nothing to me.  I am correcting duplicate segments and unclosed ways which
appear to be part of your problem as I work.  Perhaps someone can explain
what you are after in simpler terms because if we don't understand the
problem then trying to come up with ways to avoid it in the future isn't
going to happen.

Thanks John

On 17 March 2017 at 08:16, Jochen Topf <joc...@remote.org> wrote:

> Hi!
>
> I am sure there are many other problems, but can we please keep to the
> issue at hand in this thread? I am not trying to solve all problems, I
> am trying to solve one particular set of problem related to
> (multi)polygons.
>
> And I didn't pick that particular problem at random. This is something
> that affects everybody. The short of it is: Due to upcoming software
> changes, some (multi)polygons that are rendered now will disappear from
> the map or look different in the future. Again, you can find all the
> details about the problems and of what we are trying to achieve at
> http://area.jochentopf.com/ .
>
> We are having great success working on these problems. You can see the
> stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
> approach will be the best one for other problems, and it doesn't mean we
> can't do even better than we are doing until now with this problem.
>
> Btw: I am deliberately talking about "(multi)polygons". This is not
> something that only affects multipolygon relations, but all polygons,
> even the ones created from closed ways. They are simpler than mp
> relations, but that doesn't mean they can't have problems.
>
> Jochen
>
> On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:
> > Date: Thu, 16 Mar 2017 17:23:47 -0400
> > From: john whelan <jwhelan0...@gmail.com>
> > To: Jochen Topf <joc...@remote.org>
> > Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM
> >  <blake.girar...@hotosm.org>
> > Subject: Re: [HOT] Broken (multi)polygon cleanup
> >
> > This is similar to my pulling in Africa a country at a time and using
> JOSM
> > to check for errors.
> >
> > What I have noticed is one mapper will misstag then others will follow.
> So
> > sanitising the area or checking it for errors before the new HOT mappers
> > start would probably improve the overall data quality as they tend to
> > follow what is already mapped.
> >
> > Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings
> in
> > one small area by the same mapper left as area=yes.  That's quick and
> easy
> > to fix.
> >
> > Recently I came across 42 highways left untagged by the same mapper in
> the
> > same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
> >
> > Crossing highways often I'll see a town where the highway residentials
> are
> > laid out in a grid but the junctions aren't nodes.
> >
> > Another nice recent one was a batch of settlements that had been tagged
> > highway=unclassified rather than landuse=residential.  Searching for
> > highways above unclassified and crossing ways picked those out quite
> > quickly.
> >
> > These types of errors are handled more quickly directly in JOSM than
> > involving maproulette.
> >
> > Even the some other errors can be loaded up in the JOSM  to-do plugin.
> >
> > I've been cleaning up in Nigeria for a while now.  Making repeated passes
> > over the same areas over time.  I'm seeing fewer errors so my suspicion
> is
> > the improved learnOSM, tutorials and simplifying what we ask mappers to
> map
> > is paying off.  Validating new mappers work within 48 hours does pay off
> in
> > data quality terms.  Giving feedback more than two weeks later is
> usually a
> > waste of time based on my own experience and Martin's report but I don't
> > think we have the validators to do the validation within the magic 24/48
> > hours.
> >
> > Broken multipolyons do take a level of expertise to resolve.  I usually
> > avoid them.  However new mappers will fearlessly tread ahead and it is to
> > be expected they will break some.
> >
> > I don't think learnOSM goes into detail about them or about
> relationships.
> > Should they?  That's another question.
> >
> > For some errors we need boots on the ground.  Especially when it comes to
> > urban areas with limited access highways such as dual carriageways​ and a
> > highway approaching them can you turn in either direction?
> >
> > Often the most motivated people to fix the errors are the locals who will
> > be using the m

Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

Hi!

I am sure there are many other problems, but can we please keep to the
issue at hand in this thread? I am not trying to solve all problems, I
am trying to solve one particular set of problem related to
(multi)polygons.

And I didn't pick that particular problem at random. This is something
that affects everybody. The short of it is: Due to upcoming software
changes, some (multi)polygons that are rendered now will disappear from
the map or look different in the future. Again, you can find all the
details about the problems and of what we are trying to achieve at
http://area.jochentopf.com/ .

We are having great success working on these problems. You can see the
stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
approach will be the best one for other problems, and it doesn't mean we
can't do even better than we are doing until now with this problem.

Btw: I am deliberately talking about "(multi)polygons". This is not
something that only affects multipolygon relations, but all polygons,
even the ones created from closed ways. They are simpler than mp
relations, but that doesn't mean they can't have problems.

Jochen

On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:
> Date: Thu, 16 Mar 2017 17:23:47 -0400
> From: john whelan <jwhelan0...@gmail.com>
> To: Jochen Topf <joc...@remote.org>
> Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM
>  <blake.girar...@hotosm.org>
> Subject: Re: [HOT] Broken (multi)polygon cleanup
> 
> This is similar to my pulling in Africa a country at a time and using JOSM
> to check for errors.
> 
> What I have noticed is one mapper will misstag then others will follow.  So
> sanitising the area or checking it for errors before the new HOT mappers
> start would probably improve the overall data quality as they tend to
> follow what is already mapped.
> 
> Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings in
> one small area by the same mapper left as area=yes.  That's quick and easy
> to fix.
> 
> Recently I came across 42 highways left untagged by the same mapper in the
> same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
> 
> Crossing highways often I'll see a town where the highway residentials are
> laid out in a grid but the junctions aren't nodes.
> 
> Another nice recent one was a batch of settlements that had been tagged
> highway=unclassified rather than landuse=residential.  Searching for
> highways above unclassified and crossing ways picked those out quite
> quickly.
> 
> These types of errors are handled more quickly directly in JOSM than
> involving maproulette.
> 
> Even the some other errors can be loaded up in the JOSM  to-do plugin.
> 
> I've been cleaning up in Nigeria for a while now.  Making repeated passes
> over the same areas over time.  I'm seeing fewer errors so my suspicion is
> the improved learnOSM, tutorials and simplifying what we ask mappers to map
> is paying off.  Validating new mappers work within 48 hours does pay off in
> data quality terms.  Giving feedback more than two weeks later is usually a
> waste of time based on my own experience and Martin's report but I don't
> think we have the validators to do the validation within the magic 24/48
> hours.
> 
> Broken multipolyons do take a level of expertise to resolve.  I usually
> avoid them.  However new mappers will fearlessly tread ahead and it is to
> be expected they will break some.
> 
> I don't think learnOSM goes into detail about them or about relationships.
> Should they?  That's another question.
> 
> For some errors we need boots on the ground.  Especially when it comes to
> urban areas with limited access highways such as dual carriageways​ and a
> highway approaching them can you turn in either direction?
> 
> Often the most motivated people to fix the errors are the locals who will
> be using the map.  Trouble is we first have to get something in place that
> is useful to get them motivated.
> 
> So yes HOT maperthons do have a reputation of the blind leading the blind,
> ie some are led by inexperienced mappers but we are putting effort into
> reducing the number of errors.  The training group has put together a very
> good detailed set of instructions on how best to run a maperthon and if you
> can identify a problem area HOT will listen and try to address the issues.
> 
> Cheerio John
> 
> On 16 Mar 2017 3:49 pm, "Jochen Topf" <joc...@remote.org> wrote:
> 
> > Hi Blake,
> >
> > On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> > > Can you send me something that would help me see those 1600 examples
> > > you found on OSM so I can see what might be going on and how to
> > > improv

Re: [HOT] Broken (multi)polygon cleanup

[Pierre Béland]

On 16 Mar 2017 3:49 pm, "Jochen Topf"  wrote:

Hi Blake,

On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> Can you send me something that would help me see those 1600 examples
> you found on OSM so I can see what might be going on and how to
> improve things in HOT/MM workflow to reduce them?

Those 1600 are some very specific errors I am currently looking at, but
only have those on my local machine currently. But for the wider picture
you can use the OSM Inspectors "area" view to see the scope of the problem.

Just go to any place where HOT/MM was active. Here is an example:
http://tools.geofabrik.de/ osmi/?view=areas=33.65927& lat=-2.07600=8

My current workflow is as follows: I choose one particular well-defined
problem and extract the data from OSM that shows this problem. Then I
put the data into Maproulette and document the problem and how to fix
it. Some problems are easier, some harder to fix. Mappers can choose
what problems to work on based on their skill level. If somebody
doesn't feel comfortable with one of the harder problems, there are
simpler ones they can work on. I also have some more general
descriptions for the experts who want to go exploring/fixing on their
own. Once the challenges are done, I choose a new problem and start the
process from the beginning.

Here are the current challenges: http://area.jochentopf.com/ fixing.html

Usually I am splitting up larger challenges into regions, for instance
by continent or country. I could also do this by HOT activation area or
so if that makes sense for HOT mappers. Splitting up, often makes the
problems easier to fix, because problems inside an area are often very
similar, but different to problems in other areas due to mapping
priorities and methods. For instance, some regions have lots of problems
with buildings, others with landuse areas, etc. Concentrating on one
very well-defined problem at a time makes all this work simpler.

Jochen
--
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

__ _
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap. org/listinfo/hot


___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot


   ___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[john whelan]

This is similar to my pulling in Africa a country at a time and using JOSM
to check for errors.

What I have noticed is one mapper will misstag then others will follow.  So
sanitising the area or checking it for errors before the new HOT mappers
start would probably improve the overall data quality as they tend to
follow what is already mapped.

Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings in
one small area by the same mapper left as area=yes.  That's quick and easy
to fix.

Recently I came across 42 highways left untagged by the same mapper in the
same area.  The mapper had more than 2,000 edits so wasn't inexperienced.

Crossing highways often I'll see a town where the highway residentials are
laid out in a grid but the junctions aren't nodes.

Another nice recent one was a batch of settlements that had been tagged
highway=unclassified rather than landuse=residential.  Searching for
highways above unclassified and crossing ways picked those out quite
quickly.

These types of errors are handled more quickly directly in JOSM than
involving maproulette.

Even the some other errors can be loaded up in the JOSM  to-do plugin.

I've been cleaning up in Nigeria for a while now.  Making repeated passes
over the same areas over time.  I'm seeing fewer errors so my suspicion is
the improved learnOSM, tutorials and simplifying what we ask mappers to map
is paying off.  Validating new mappers work within 48 hours does pay off in
data quality terms.  Giving feedback more than two weeks later is usually a
waste of time based on my own experience and Martin's report but I don't
think we have the validators to do the validation within the magic 24/48
hours.

Broken multipolyons do take a level of expertise to resolve.  I usually
avoid them.  However new mappers will fearlessly tread ahead and it is to
be expected they will break some.

I don't think learnOSM goes into detail about them or about relationships.
Should they?  That's another question.

For some errors we need boots on the ground.  Especially when it comes to
urban areas with limited access highways such as dual carriageways​ and a
highway approaching them can you turn in either direction?

Often the most motivated people to fix the errors are the locals who will
be using the map.  Trouble is we first have to get something in place that
is useful to get them motivated.

So yes HOT maperthons do have a reputation of the blind leading the blind,
ie some are led by inexperienced mappers but we are putting effort into
reducing the number of errors.  The training group has put together a very
good detailed set of instructions on how best to run a maperthon and if you
can identify a problem area HOT will listen and try to address the issues.

Cheerio John

On 16 Mar 2017 3:49 pm, "Jochen Topf"  wrote:

> Hi Blake,
>
> On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> > Can you send me something that would help me see those 1600 examples
> > you found on OSM so I can see what might be going on and how to
> > improve things in HOT/MM workflow to reduce them?
>
> Those 1600 are some very specific errors I am currently looking at, but
> only have those on my local machine currently. But for the wider picture
> you can use the OSM Inspectors "area" view to see the scope of the problem.
>
> Just go to any place where HOT/MM was active. Here is an example:
> http://tools.geofabrik.de/osmi/?view=areas=33.65927;
> lat=-2.07600=8
>
> My current workflow is as follows: I choose one particular well-defined
> problem and extract the data from OSM that shows this problem. Then I
> put the data into Maproulette and document the problem and how to fix
> it. Some problems are easier, some harder to fix. Mappers can choose
> what problems to work on based on their skill level. If somebody
> doesn't feel comfortable with one of the harder problems, there are
> simpler ones they can work on. I also have some more general
> descriptions for the experts who want to go exploring/fixing on their
> own. Once the challenges are done, I choose a new problem and start the
> process from the beginning.
>
> Here are the current challenges: http://area.jochentopf.com/fixing.html
>
> Usually I am splitting up larger challenges into regions, for instance
> by continent or country. I could also do this by HOT activation area or
> so if that makes sense for HOT mappers. Splitting up, often makes the
> problems easier to fix, because problems inside an area are often very
> similar, but different to problems in other areas due to mapping
> priorities and methods. For instance, some regions have lots of problems
> with buildings, others with landuse areas, etc. Concentrating on one
> very well-defined problem at a time makes all this work simpler.
>
> Jochen
> --
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/
> +49-351-31778688
>
> ___
> HOT mailing list
> 

Re: [HOT] Broken (multi)polygon cleanup

[Jochen Topf]

Hi Blake,

On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> Can you send me something that would help me see those 1600 examples
> you found on OSM so I can see what might be going on and how to
> improve things in HOT/MM workflow to reduce them?

Those 1600 are some very specific errors I am currently looking at, but
only have those on my local machine currently. But for the wider picture
you can use the OSM Inspectors "area" view to see the scope of the problem.

Just go to any place where HOT/MM was active. Here is an example:
http://tools.geofabrik.de/osmi/?view=areas=33.65927=-2.07600=8

My current workflow is as follows: I choose one particular well-defined
problem and extract the data from OSM that shows this problem. Then I
put the data into Maproulette and document the problem and how to fix
it. Some problems are easier, some harder to fix. Mappers can choose
what problems to work on based on their skill level. If somebody
doesn't feel comfortable with one of the harder problems, there are
simpler ones they can work on. I also have some more general
descriptions for the experts who want to go exploring/fixing on their
own. Once the challenges are done, I choose a new problem and start the
process from the beginning.

Here are the current challenges: http://area.jochentopf.com/fixing.html

Usually I am splitting up larger challenges into regions, for instance
by continent or country. I could also do this by HOT activation area or
so if that makes sense for HOT mappers. Splitting up, often makes the
problems easier to fix, because problems inside an area are often very
similar, but different to problems in other areas due to mapping
priorities and methods. For instance, some regions have lots of problems
with buildings, others with landuse areas, etc. Concentrating on one
very well-defined problem at a time makes all this work simpler.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Broken (multi)polygon cleanup

[Blake Girardot HOT/OSM]

Hi Jochen,

Thank you very much for writing to the HOT list!

I think you will find a lot of enthusiasm for helping get these fixed
as well as improving our training and instructions to make sure we
minimize new mappers who come to OSM through HOT and Missing Maps
making these errors going forward.

Can you send me something that would help me see those 1600 examples
you found on OSM so I can see what might be going on and how to
improve things in HOT/MM workflow to reduce them?

Cheers,
Blake

On Wed, Mar 15, 2017 at 4:06 PM, Jochen Topf  wrote:
> Hi!
>
> There are many broken (multi)polygon geometries in OSM data leading to
> all sorts of problems. I have started an effort to get this cleaned up.
> You can read more about this at http://area.jochentopf.com/ .
>
> This effort has been running for about a month now and we can already see
> some good results. See 
> https://github.com/osmlab/fixing-polygons-in-osm/issues/15
> for some news about what's been happening.
>
> I am writing to the HOT list today, because a not-so-small part of the
> problems come from HOT mappers. For instance, looking at one particular
> problem here (spikes in closed ways), I am seeing about 5800 errors, of
> which about 1600 are in areas covered by the HOT tasking manager. (Of
> couse not all of these have been done in the course of HOT mapping, but
> probably most of them.)
>
> I am reaching out to some of the regional OSM communities and also to
> HOT, because I want to a) ask for the help of all the communities in OSM
> to fix those errors and b) generally raise awareness, so those problems
> can be minimized in the future.
>
> Please go to the web sites linked above and read up on what the problems
> are and how we are fixing them. I also want to offer my help in
> preparing special data extracts or Maproulette challenges etc. to help
> the HOT community participate in the cleaning up effort in a way that
> makes sense to you. I could, for instance, put those 1600 errors
> mentioned above in a special Maproulette challenge, if that would be a
> good way for HOT mappers to work on them.
>
> Tell me what you think. I appreciate any feedback.
>
> Jochen
> --
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688
>
> ___
> HOT mailing list
> HOT@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/hot



-- 

Blake Girardot
Humanitarian OpenStreetMap Team, TM3 Project Manager
skype: jblakegirardot
HOT Core Team Contact: i...@hotosm.org
Live OSM Mapper-Support channel - https://hotosm-slack.herokuapp.com/
BE A PART OF HOT'S MICRO GRANTS: https://donate.hotosm.org/

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

[HOT] Broken (multi)polygon cleanup

[Jochen Topf]

Hi!

There are many broken (multi)polygon geometries in OSM data leading to
all sorts of problems. I have started an effort to get this cleaned up.
You can read more about this at http://area.jochentopf.com/ .

This effort has been running for about a month now and we can already see
some good results. See 
https://github.com/osmlab/fixing-polygons-in-osm/issues/15
for some news about what's been happening.

I am writing to the HOT list today, because a not-so-small part of the
problems come from HOT mappers. For instance, looking at one particular
problem here (spikes in closed ways), I am seeing about 5800 errors, of
which about 1600 are in areas covered by the HOT tasking manager. (Of
couse not all of these have been done in the course of HOT mapping, but
probably most of them.)

I am reaching out to some of the regional OSM communities and also to
HOT, because I want to a) ask for the help of all the communities in OSM
to fix those errors and b) generally raise awareness, so those problems
can be minimized in the future.

Please go to the web sites linked above and read up on what the problems
are and how we are fixing them. I also want to offer my help in
preparing special data extracts or Maproulette challenges etc. to help
the HOT community participate in the cleaning up effort in a way that
makes sense to you. I could, for instance, put those 1600 errors
mentioned above in a special Maproulette challenge, if that would be a
good way for HOT mappers to work on them.

Tell me what you think. I appreciate any feedback.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot