Re: [OSM-talk] Fixing broken multipolygons, some notes

[Christoph Hormann]

On Monday 20 March 2017, Andreas Vilén wrote:
>
> It's not up to me to decide if this data is to be deleted or not. If
> you want to do that, raise the question with each respective
> country's mailing list.

I don't want to push the issue - and there is little chance for such a 
suggestion without significant parts of the local community feeling a 
need for this on their own.

Since I am pretty sure quite a few of the mappers from Sweden and 
Finland read this list people are likely aware of the problem now 
(likely were before in some way as well).

Despite this the whole matter also has a more global component of 
course.  Mapping of the large wooded areas of the planet is a problem 
in OSM that is going to bother us for the forseeable future.  Right now 
the only areas where you can produce larger area maps with forest 
rendering in good quality based on OSM data alone are western and 
central Europe (mostly, there are gaps even in that, in particular in 
the Italian Alps).  And these are obviously not particularly difficult 
in terms of forest mapping.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Jochen Topf]

On Mon, Mar 20, 2017 at 07:15:47AM +0100, Jochen Topf wrote:
> On Mon, Mar 20, 2017 at 06:27:05AM +0100, Andreas Vilén wrote:
> > It's not up to me to decide if this data is to be deleted or not. If
> > you want to do that, raise the question with each respective country's
> > mailing list.
> 
> I was under the impression that were in contact with the Swedish
 ^you

> community on this topic and that the Swedish community had decided they
> wanted to keep the data. So I thought this issue was settled.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Jochen Topf]

On Mon, Mar 20, 2017 at 06:27:05AM +0100, Andreas Vilén wrote:
> It's not up to me to decide if this data is to be deleted or not. If
> you want to do that, raise the question with each respective country's
> mailing list.

I was under the impression that were in contact with the Swedish
community on this topic and that the Swedish community had decided they
wanted to keep the data. So I thought this issue was settled.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Andreas Vilén]

There is a difference between data errors that the osmi will detect and bad 
quality map data. Corine is the latter.

It's not up to me to decide if this data is to be deleted or not. If you want 
to do that, raise the question with each respective country's mailing list.

/Andreas

Skickat från min iPhone

> 19 mars 2017 kl. 19:32 skrev Christoph Hormann :
> 
>> On Sunday 19 March 2017, Andreas Vilén wrote:
>> 
>> Also, as has been pointed out earlier, Corine data might be bad, but
>> does not contain that many pure data errors as we define them.
> 
> That is not quite accurate in my experience.  As i explained in
> 
> https://lists.openstreetmap.org/pipermail/talk/2017-February/077553.html
> 
> landcover mappings like corine are based on selecting the least unlikely 
> of a fixed set of landcover classes at a certain scale based on certain 
> criteria and reference areas and this frequently produces completely 
> bogus results.
> 
> In areas like here:
> 
> http://www.openstreetmap.org/#map=13/56.7935/16.0168
> http://www.openstreetmap.org/#map=13/62.7015/25.6678
> 
> the Corine based landcover data in my eyes has not connection to reality 
> at all, it is factually simply wrong.
> 
> I understand that the perspective to loose all the data and to be 
> without any forest mapping until manual mapping has filled the gaps 
> seems not so pleasant but i am pretty sure there are a lot of people 
> from abroad who would be glad to help you with either manual forest 
> mapping or importing better quality data in a way that is better 
> maintainable and more compatible with manual mapping activity.
> 
> However as long as the bad quality Corine data is there and meant to 
> stay few people are interested in editing this mostly meaningless data.
> 
> -- 
> Christoph Hormann
> http://www.imagico.de/
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Christoph Hormann]

On Sunday 19 March 2017, Andreas Vilén wrote:
>
> Also, as has been pointed out earlier, Corine data might be bad, but
> does not contain that many pure data errors as we define them.

That is not quite accurate in my experience.  As i explained in

https://lists.openstreetmap.org/pipermail/talk/2017-February/077553.html

landcover mappings like corine are based on selecting the least unlikely 
of a fixed set of landcover classes at a certain scale based on certain 
criteria and reference areas and this frequently produces completely 
bogus results.

In areas like here:

http://www.openstreetmap.org/#map=13/56.7935/16.0168
http://www.openstreetmap.org/#map=13/62.7015/25.6678

the Corine based landcover data in my eyes has not connection to reality 
at all, it is factually simply wrong.

I understand that the perspective to loose all the data and to be 
without any forest mapping until manual mapping has filled the gaps 
seems not so pleasant but i am pretty sure there are a lot of people 
from abroad who would be glad to help you with either manual forest 
mapping or importing better quality data in a way that is better 
maintainable and more compatible with manual mapping activity.

However as long as the bad quality Corine data is there and meant to 
stay few people are interested in editing this mostly meaningless data.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Mattias Dalkvist]

On Sun, Mar 19, 2017 at 12:09 PM, Martin Koppenhoefer <
dieterdre...@gmail.com> wrote:

>
> On 18 Mar 2017, at 21:40, Sandor Seres  wrote:
>
> In another style, typical land related names are on the water like here
> http://osm.org/go/0Tt1PZIt-?layers=T .
>
>
> seems like either a bad import or localities on the sea, e.g. here:
> http://www.openstreetmap.org/node/2535180885
> 
>
>

The names in that case are bays and islands but it looks like there are 2
lake relations and only one of them have the islands as inner members
http://www.openstreetmap.org/relation/175607#map=14/60.4445/10.2434
http://www.openstreetmap.org/relation/2922000#map=14/60.4435/10.2359

-- 
Dalkvist
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[James]

Also if I remember correctly the whole point was to optimize/be able to
process osm data faster by not having to deal with so many errors(each case
can slow down the processing)

On Mar 19, 2017 7:23 AM, "Martin Koppenhoefer" 
wrote:

>
>
> sent from a phone
>
> > On 19 Mar 2017, at 10:03, Andreas Vilén  wrote:
> >
> > The main problem with Corine is that oftentimes the landuse data
> overlaps villages (which I found when I mapped mountain villages in
> southern Spain last week as well)
>
>
> whenever looking closeup at any corine data it was hardly possible to see
> the same borders they have in the aerial imagery. Often they resemble, but
> never (in the cases I looked at) the border was drawn where I'd expect a
> human mapper to draw it. The data doesn't seem suitable for small scale
> maps (small scale is what you typically observe on the ground)
>
> Cheers,
> Martin
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Martin Koppenhoefer]

sent from a phone

> On 19 Mar 2017, at 10:03, Andreas Vilén  wrote:
> 
> The main problem with Corine is that oftentimes the landuse data overlaps 
> villages (which I found when I mapped mountain villages in southern Spain 
> last week as well)


whenever looking closeup at any corine data it was hardly possible to see the 
same borders they have in the aerial imagery. Often they resemble, but never 
(in the cases I looked at) the border was drawn where I'd expect a human mapper 
to draw it. The data doesn't seem suitable for small scale maps (small scale is 
what you typically observe on the ground)

Cheers,
Martin 
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Martin Koppenhoefer]

sent from a phone

> On 18 Mar 2017, at 21:40, Sandor Seres  wrote:
> 
> In another style, typical land related names are on the water like here 
> http://osm.org/go/0Tt1PZIt-?layers=T .


seems like either a bad import or localities on the sea, e.g. here: 
http://www.openstreetmap.org/node/2535180885

Cheers,
Martin ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Andreas Vilén]

To be fair, "the local community" in this case is probably mostly me and my
interpretation of the Swedish community's reaction when someone tried to
remove Corine imagery rather carelessly (introduced a lot of new errors) a
couple of years ago. It's too bad that we're talking about a region when no
one from that region is posting in this mailing list (even though I'm sure
many read it).

Also, as has been pointed out earlier, Corine data might be bad, but does
not contain that many pure data errors as we define them. This is
approximately the area in Sandor's link in OSM inspector:
http://tools.geofabrik.de/osmi/?view=areas=10.35829=60.44245=10
This is probably the worst problem area but that has been mapped by hand:
http://tools.geofabrik.de/osmi/?view=areas=14.63086=56.93988=9

The main problem with Corine is that oftentimes the landuse data overlaps
villages (which I found when I mapped mountain villages in southern Spain
last week as well) and also when someone has been mapping by hand and
doesn't clean up the Corine data while doing that (I have been doing a lot
of cleanup here, but lots of fixing remains, but as you can see, no mapping
errors:
http://tools.geofabrik.de/osmi/?view=areas=16.63835=56.73536=12
).

Maybe new error tools can be developed to find for example when Corine data
overlaps other landuse data, or it can be taught to analyze imagery for
buildings inside farmland/forest tags?

/Andreas

On Sat, Mar 18, 2017 at 11:24 PM, Frederik Ramm  wrote:

> Sandor,
>
>if I understand you correctly your basic message is "let's try and
> improve osm2pgsql's polygon interpretation rules instead of fixing the
> data", or at least "while we wait for the data fixing to be completed".
>
> I think this is not a good idea because, as you remark yourself, those
> working with the OSM source data directly will not profit from more
> magic in osm2pgsql. We would have more situations where someone
> processes OSM data with, say, QGIS and thinks "uh, I must have done
> something wrong, because on the OSM map there's a forest here and on my
> map there isn't".
>
> Fixing the data in OSM is the better approach. In many cases, broken
> polygons are a result of a broken import or careless editing and if
> MapRoulette prompts someone to open their editor in that location, they
> are likely to find quite a few other things worth of attention.
>
> > The “Fixing broken polygons”, especially programmatic/mass fixing, could
> > be more dangerous to all users.
>
> I wholly agree that, with perhaps a very few narrowly defined
> exceptions, nobody should make an attempt to fix broken polygons
> automatically because many things go wrong (and the automatic fix would
> not give us the advantage I mentioned above, that someone looking at the
> data in the area might find other things amiss).
>
> > Let us look at a map extract from the
> > mentioned Scandinavian forests
>
> It has been suggested to completely remove some CORINE-based landcover
> import in Scandinavia on the basis that not only there are many broken
> polygons, but also that it bears very little resemblance to the reality
> on the ground. People have said it is a nicely painted map but not much
> more. But I believe the local community said they preferred a nicely
> painted and somewhat wrong map over having to trace the forests from
> aerial imagery themselves ;)
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Frederik Ramm]

Sandor,

   if I understand you correctly your basic message is "let's try and
improve osm2pgsql's polygon interpretation rules instead of fixing the
data", or at least "while we wait for the data fixing to be completed".

I think this is not a good idea because, as you remark yourself, those
working with the OSM source data directly will not profit from more
magic in osm2pgsql. We would have more situations where someone
processes OSM data with, say, QGIS and thinks "uh, I must have done
something wrong, because on the OSM map there's a forest here and on my
map there isn't".

Fixing the data in OSM is the better approach. In many cases, broken
polygons are a result of a broken import or careless editing and if
MapRoulette prompts someone to open their editor in that location, they
are likely to find quite a few other things worth of attention.

> The “Fixing broken polygons”, especially programmatic/mass fixing, could
> be more dangerous to all users.

I wholly agree that, with perhaps a very few narrowly defined
exceptions, nobody should make an attempt to fix broken polygons
automatically because many things go wrong (and the automatic fix would
not give us the advantage I mentioned above, that someone looking at the
data in the area might find other things amiss).

> Let us look at a map extract from the
> mentioned Scandinavian forests

It has been suggested to completely remove some CORINE-based landcover
import in Scandinavia on the basis that not only there are many broken
polygons, but also that it bears very little resemblance to the reality
on the ground. People have said it is a nicely painted map but not much
more. But I believe the local community said they preferred a nicely
painted and somewhat wrong map over having to trace the forests from
aerial imagery themselves ;)

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[john whelan]

There has been some discussion on the HOT mailing list that makes things a
bit clearer.

OSM in general has a fair number of things that have been added in a less
than ideal way.  It can be difficult to correct some things as we have
guidelines or recommended practises as opposed to hard and fast rules but
maproulette has managed to identify a number of areas where there is some
agreement about what needs to be corrected.

JOSM validation also tries to identify problem areas so I suspect fixing
the underlying data is the better long term solution rather than ensuring
all the different rendering systems are more robust.  Robustness costs
machine cycles​ as well.

Cheerio John

On 18 Mar 2017 4:43 pm, "Sandor Seres"  wrote:

> I am new to this list and therefore apologize for eventual
> misinterpretations and wrong stile. The motivation for the mail is a
> worrying mail on the local list about the purer osm2pgsql based maps and
> about the “broken polygons” fixing strategies. The mentioned white spots in
> the Scandinavian forests are just an illustration. By simply dropping
> broken polygons, empty spots will be typical for any area types and for any
> corners of the Planet.
>
> As I understand, osm2pgsql is an application doing data preparation from
> the OSM source data up to a DB used by many mapmakers for rendering. We can
> see that almost all OSM based public mapping system use this database and
> consequently repeat the same anomalies. Therefore, maybe, making the
> osm2pgsql more robust could be a better strategy. There is still a large
> potential for such strengthening. Just waiting for “do-ocracy” reparations
> is really a long-term strategy. Anyway, users starting from the source OSM
> data will not be affected by any of these strategies.
>
> The “Fixing broken polygons”, especially programmatic/mass fixing, could
> be more dangerous to all users. Just look at the many possible
> self-crossing fixing options. Loosely defined notions open for different
> interpretations and different sets of error criteria. Consequently, for the
> same object type we may have (and we do) different error classes and
> reparation tools. Besides the typical polygon interpretations as area (ESRI
> polygon redefinition) or as a closed polygonal line, we simply can’t find
> in the documentation what “outer”, “inner”, “hole” … notions actually mean.
> The interpretation (individual perception) of these notions is left to us
> and there we have a source of misunderstandings. For instance, if we assume
> that “outer” border polygons define the interior candidate points (points
> inside and on the border) and “inner” border polygons define (in the same
> way) exterior points of area than self-crossings, touching polygons,
> polygon overlaps, crossings… are not errors at all.
>
> However, my point here is still something else. The “broken multipolygon”
> (whatever that means) issue is just “the tip of the iceberg”. There is
> still remaining huge number of anomalies caused by area object relations
> from different area classes. I intentionally use the anomaly notion, as a
> moderate form for error, because many people/mapmakers may liv with them.
> But a modern GIS system and a vector layers based digital cartography
> cannot tolerate them. Let me present some arguments and illustrations. Let
> us look at a map extract from the mentioned Scandinavian forests here
> http://osm.org/go/0Tt1PZIt- . The example could be taken from any corner
> of the Planet and, as mentioned, there is huge number of similar cases. At
> the first glance, everything looks correct and nice (and it is). However,
> we see immediately that something is still wrong. The forest type symbols
> are placed directly over the water. In another style, typical land related
> names are on the water like here http://osm.org/go/0Tt1PZIt-?layers=T .
> Looking at the source data we can see that the lake in the middle is placed
> over an empty space (intentionally, not a hole) where the border of the
> lake runs slightly in and outside the forests. At the same time, we can see
> many forest areas inside the mentioned empty space overwritten with the
> lake that has no holes. Consequently, there are many missing islands in the
> lake and many missing forest areas in the extract. Note that only on that
> little extract there are more than 40 of the described anomalies. What
> more, there are many lakes with borders running in/out of forest areas
> (corridor border overlaps), having considerable parts over a forest and
> holes in forests, partly overlapping several disjunctive forest areas and
> so on, and the contrary. Extending the case to the Planet and other area
> types combinations we may feel the extent of the issue. There were attempts
> to compensate these problems in renderings like rendering the holes,
> rendering smaller over larger objects and so on. These actions generally do
> not work. Simply, they do good some places and damaging 

Re: [OSM-talk] Fixing broken multipolygons

[Martin Koppenhoefer]

sent from a phone

> On 4 Mar 2017, at 16:54, Jochen Topf  wrote:
> 
> the old-style
> polygons are next and I will create challenges for them, too.


great, they're really legacy to get rid of.

cheers,
Martin 

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Sat, Mar 04, 2017 at 11:03:26AM +, molto...@gmail.com wrote:
> You should also look out for MPs with tags on the outer ring but
> should actually only be on the realtion. Having the same tags on inner
> and outer is a nice heuristic that QA tools detect, but is not the
> only way that old-style polygons (which AFAIU wont be supported by
> osm2pgsql at some stage) can happen.

I do.

https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/problems.md#old-style-tagging
http://area.jochentopf.com/stats/#old_style_multipolygons

Same tags on inner and outer is just a subset of the old-style polygons.

Currently I am concentrating on actually broken polygons, the old-style
polygons are next and I will create challenges for them, too.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Sat, Mar 04, 2017 at 10:50:41AM +0100, Martin Koppenhoefer wrote:
> > On 4 Mar 2017, at 08:49, Jochen Topf  wrote:
> > 
> > Looking at the graphs on http://area.jochentopf.com/stats you can see
> > that the number of (multi)polygons is growing steadily, while the number
> > of errors has been more or less flat over the last half year.
> 
> 
> nice stats and good results.
> I want to point out though that "
> Errors: Inner rings with same tags as outer rings"
> 
> are not necessarily errors, and should not be "fixed" unless you know
> very well the situation and can tell that there's indeed a
> redundancy/data problem. E.g. you can have a building or building:part
> inside another building or building:part. These could be tagged still
> "incompletely" hence having just the same tags for the moment but be
> different objects nonetheless. Similarly woods inside woods, etc.

"Inner rings with same tags as outer rings" is a subset of old-style
multipolygons. Well, as you correctly point out, some of them might be
okay, but most of them are especially "bad" cases of old-style
multipolygons. And they are another good reason why we need to get rid
of old-style multipolygons. There is just no way to figure out
automatically, what's right here and what's not. So humans have to fix
those. I'll create Maproulette challenges for these too at some point.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[moltonel]

On 4 March 2017 9:50:41 AM GMT+00:00, Martin Koppenhoefer 
>I want to point out though that "
>Errors: Inner rings with same tags as outer rings"
>
>are not necessarily errors, and should not be "fixed" unless you know
>very well the situation and can tell that there's indeed a
>redundancy/data problem. E.g. you can have a building or building:part
>inside another building or building:part. These could be tagged still
>"incompletely" hence having just the same tags for the moment but be
>different objects nonetheless. Similarly woods inside woods, etc.
>


You should also look out for MPs with tags on the outer ring but should 
actually only be on the realtion. Having the same tags on inner and outer is a 
nice heuristic that QA tools detect, but is not the only way that old-style 
polygons (which AFAIU wont be supported by osm2pgsql at some stage) can happen.

Sometimes it's tricky: I recently fixed a MP with no tag, 1 inner, 1 outer, and 
the outer taged as natural=wood leisure=park. Turned out the park was mostly 
wood with a clearing, so moved only natural=wood to the relation.


-- 
Sent from my internet

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Martin Koppenhoefer]

sent from a phone

> On 4 Mar 2017, at 08:49, Jochen Topf  wrote:
> 
> Looking at the graphs on http://area.jochentopf.com/stats you can see
> that the number of (multi)polygons is growing steadily, while the number
> of errors has been more or less flat over the last half year.


nice stats and good results.
I want to point out though that "
Errors: Inner rings with same tags as outer rings"

are not necessarily errors, and should not be "fixed" unless you know very well 
the situation and can tell that there's indeed a redundancy/data problem. E.g. 
you can have a building or building:part inside another building or 
building:part. These could be tagged still "incompletely" hence having just the 
same tags for the moment but be different objects nonetheless. Similarly woods 
inside woods, etc.

Cheers,
Martin ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

Hi!

You all have been very busy fixing my challenges, so here are some more:
http://area.jochentopf.com/fixing.html#self-intersecting-polygon-ways

I have started to keep an ongoing "blog" where I'll post news and
information about the effort to fix the (multi)polygons:

https://github.com/osmlab/fixing-polygons-in-osm/issues/15

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Fri, Mar 03, 2017 at 05:27:18PM +0100, Sebastiaan Couwenberg wrote:
> On 03/03/2017 05:10 PM, m...@rtijn.org wrote:
> > Since the ‘self-intersecting’ challenge is now complete I featured the 
> > ‘Wrong role’ challenge in MapRoulette. Happy fixing!
> 
> Are there plans to make these challenges permanent or periodically
> re-introduce them when a new batch of issues has been prepared?

No plans yet. My current focus is to get rid of all the old cases. Once
that is (mostly) done, we can see how things develop.

> These are very common issues that will be introduced by mappers in the
> future.

Looking at the graphs on http://area.jochentopf.com/stats you can see
that the number of (multi)polygons is growing steadily, while the number
of errors has been more or less flat over the last half year. So either
there are not a significant amount of new problems or old problems are
being fixed as fast as new problems are introduced. Both leads me to
believe that existing QA tools and/or better editors than we had
historically keep this problem in check. But, as I said, we'll first
need to get the number of errors down to low level and see how things
look then.

And maybe when there are less cases we can better see specific problems
creeping up that can be solved by improving editors etc.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Fri, Mar 03, 2017 at 09:07:37AM -0800, Clifford Snow wrote:
> I noticed a number of riverbanks with self-intersecting ways in the PNW
> that appear on OSMI. How do I go about creating a challenge to fix them?

The challenges I have posted recently are only the beginning. I have
more challenges in the works that will cover all multipolygon problems.
This is just a matter of me rolling out those challenges a few at a time
so the community can concentrate on one problem before tackling the next
one. 

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[m]

Jochen Topf now has technology in place that does this specifically for 
buildings. Perhaps he can do the same for other types and help you publish?

Martijn

> On Mar 3, 2017, at 10:07 AM, Clifford Snow  wrote:
> 
> Martijn,
> I noticed a number of riverbanks with self-intersecting ways in the PNW that 
> appear on OSMI. How do I go about creating a challenge to fix them?
> 
> Clifford
> 
> On Fri, Mar 3, 2017 at 8:10 AM, > 
> wrote:
> Since the ‘self-intersecting’ challenge is now complete I featured the ‘Wrong 
> role’ challenge in MapRoulette. Happy fixing!
> Martijn
> 
>> On Feb 25, 2017, at 2:04 AM, Manfred A. Reiter > > wrote:
>> 
>> Hi Martijn,
>> 
>> 2017-02-24 23:35 GMT+01:00  >:
>> I made the challenge ‘Featured’ in MapRoulette so it becomes easier to spot.
>> What a mess in some places I looked at…
>> 
>> +1
>> 
>> some of them are here: https://wiki.openstreetmap.org/wiki/Extremmapping 
>> 
>> feel free to add yours as well :)
>>  
>> Please help fix!
>> 
>> YES!
>>  
>> Martijn
>> 
>> > On Feb 22, 2017, at 1:07 AM, Jochen Topf > > > wrote:
>> >
>> > On Tue, Feb 21, 2017 at 07:44:18PM +0100, Sebastiaan Couwenberg wrote:
>> >> On 02/21/2017 05:40 PM, Jochen Topf wrote:
>> >>> Find all challenges and instructions here:
>> >>> http://area.jochentopf.com/fixing.html 
>> >>> 
>> >>
>> >> My OCD complains about the typo before the challenge links, please do
>> >>
>> >> sed -i 's/Got to /Go to/g' fixing.html
>> >>
>> >> Also the Maproulette paragraph is no longer accurate now that more than
>> >> one challenge has been created.
>> >
>> > Fixed. Thank you!
>> >
>> > Jochen
>> > --
>> > Jochen Topf  joc...@remote.org   
>> > https://www.jochentopf.com/   
>> > +49-351-31778688 
>> >
>> > ___
>> > talk mailing list
>> > talk@openstreetmap.org 
>> > https://lists.openstreetmap.org/listinfo/talk 
>> > 
>> 
>> 
>> ___
>> talk mailing list
>> talk@openstreetmap.org 
>> https://lists.openstreetmap.org/listinfo/talk 
>> 
>> 
>> 
>> 
>> -- 
>> ## Manfred Reiter - -
>> ## N49° 25' 11.028" E6° 50' 47.328" 
>> ## www.weeklyOSM.eu 
> 
> ___
> talk mailing list
> talk@openstreetmap.org 
> https://lists.openstreetmap.org/listinfo/talk 
> 
> 
> 
> 
> 
> -- 
> @osm_seattle
> osm_seattle.snowandsnow.us 
> OpenStreetMap: Maps with a human touch

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Clifford Snow]

Martijn,
I noticed a number of riverbanks with self-intersecting ways in the PNW
that appear on OSMI. How do I go about creating a challenge to fix them?

Clifford

On Fri, Mar 3, 2017 at 8:10 AM,  wrote:

> Since the ‘self-intersecting’ challenge is now complete I featured the
> ‘Wrong role’ challenge in MapRoulette. Happy fixing!
> Martijn
>
> On Feb 25, 2017, at 2:04 AM, Manfred A. Reiter 
> wrote:
>
> Hi Martijn,
>
> 2017-02-24 23:35 GMT+01:00  :
>
>> I made the challenge ‘Featured’ in MapRoulette so it becomes easier to
>> spot.
>> What a mess in some places I looked at…
>
>
> +1
>
> some of them are here: https://wiki.openstreetmap.org/wiki/Extremmapping
> feel free to add yours as well :)
>
>
>> Please help fix!
>>
>
> YES!
>
>
>> Martijn
>>
>> > On Feb 22, 2017, at 1:07 AM, Jochen Topf  wrote:
>> >
>> > On Tue, Feb 21, 2017 at 07:44:18PM +0100, Sebastiaan Couwenberg wrote:
>> >> On 02/21/2017 05:40 PM, Jochen Topf wrote:
>> >>> Find all challenges and instructions here:
>> >>> http://area.jochentopf.com/fixing.html
>> >>
>> >> My OCD complains about the typo before the challenge links, please do
>> >>
>> >> sed -i 's/Got to /Go to/g' fixing.html
>> >>
>> >> Also the Maproulette paragraph is no longer accurate now that more than
>> >> one challenge has been created.
>> >
>> > Fixed. Thank you!
>> >
>> > Jochen
>> > --
>> > Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-
>> 351-31778688
>> >
>> > ___
>> > talk mailing list
>> > talk@openstreetmap.org
>> > https://lists.openstreetmap.org/listinfo/talk
>>
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
>
>
> --
> ## Manfred Reiter - -
> ## N49° 25' 11.028" E6° 50' 47.328"
> ## www.weeklyOSM.eu 
>
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>


-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Michael Reichert]

Hi,

Am 03.03.2017 um 17:27 schrieb Sebastiaan Couwenberg:
> Are there plans to make these challenges permanent or periodically
> re-introduce them when a new batch of issues has been prepared?

The German speaking countries have a handful of mappers who regularly
look for multipolygon errors using OSMI. Just have a look at the Areas
view of OSMI at the border of Germany and the Netherlands.

http://tools.geofabrik.de/osmi/?view=areas=6.68088=52.16821=9

Best regards

Michael



-- 
Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten
ausgenommen)
I prefer GPG encryption of emails. (does not apply on mailing lists)



signature.asc
Description: OpenPGP digital signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Sebastiaan Couwenberg]

On 03/03/2017 05:10 PM, m...@rtijn.org wrote:
> Since the ‘self-intersecting’ challenge is now complete I featured the ‘Wrong 
> role’ challenge in MapRoulette. Happy fixing!

Are there plans to make these challenges permanent or periodically
re-introduce them when a new batch of issues has been prepared?

These are very common issues that will be introduced by mappers in the
future.

Having Maproulette challeges to fix the issues also visualized by the
OSMI Area maps is a great way to help fix these QA issues.

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[m]

Since the ‘self-intersecting’ challenge is now complete I featured the ‘Wrong 
role’ challenge in MapRoulette. Happy fixing!
Martijn

> On Feb 25, 2017, at 2:04 AM, Manfred A. Reiter  wrote:
> 
> Hi Martijn,
> 
> 2017-02-24 23:35 GMT+01:00  >:
> I made the challenge ‘Featured’ in MapRoulette so it becomes easier to spot.
> What a mess in some places I looked at…
> 
> +1
> 
> some of them are here: https://wiki.openstreetmap.org/wiki/Extremmapping 
> 
> feel free to add yours as well :)
>  
> Please help fix!
> 
> YES!
>  
> Martijn
> 
> > On Feb 22, 2017, at 1:07 AM, Jochen Topf  > > wrote:
> >
> > On Tue, Feb 21, 2017 at 07:44:18PM +0100, Sebastiaan Couwenberg wrote:
> >> On 02/21/2017 05:40 PM, Jochen Topf wrote:
> >>> Find all challenges and instructions here:
> >>> http://area.jochentopf.com/fixing.html 
> >>> 
> >>
> >> My OCD complains about the typo before the challenge links, please do
> >>
> >> sed -i 's/Got to /Go to/g' fixing.html
> >>
> >> Also the Maproulette paragraph is no longer accurate now that more than
> >> one challenge has been created.
> >
> > Fixed. Thank you!
> >
> > Jochen
> > --
> > Jochen Topf  joc...@remote.org   
> > https://www.jochentopf.com/   +49-351-31778688 
> > 
> >
> > ___
> > talk mailing list
> > talk@openstreetmap.org 
> > https://lists.openstreetmap.org/listinfo/talk 
> > 
> 
> 
> ___
> talk mailing list
> talk@openstreetmap.org 
> https://lists.openstreetmap.org/listinfo/talk 
> 
> 
> 
> 
> -- 
> ## Manfred Reiter - -
> ## N49° 25' 11.028" E6° 50' 47.328" 
> ## www.weeklyOSM.eu 
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[m]

I made the challenge ‘Featured’ in MapRoulette so it becomes easier to spot. 
What a mess in some places I looked at…Please help fix!
Martijn

> On Feb 22, 2017, at 1:07 AM, Jochen Topf  wrote:
> 
> On Tue, Feb 21, 2017 at 07:44:18PM +0100, Sebastiaan Couwenberg wrote:
>> On 02/21/2017 05:40 PM, Jochen Topf wrote:
>>> Find all challenges and instructions here:
>>> http://area.jochentopf.com/fixing.html
>> 
>> My OCD complains about the typo before the challenge links, please do
>> 
>> sed -i 's/Got to /Go to/g' fixing.html
>> 
>> Also the Maproulette paragraph is no longer accurate now that more than
>> one challenge has been created.
> 
> Fixed. Thank you!
> 
> Jochen
> -- 
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Tue, Feb 21, 2017 at 07:44:18PM +0100, Sebastiaan Couwenberg wrote:
> On 02/21/2017 05:40 PM, Jochen Topf wrote:
> > Find all challenges and instructions here:
> > http://area.jochentopf.com/fixing.html
> 
> My OCD complains about the typo before the challenge links, please do
> 
>  sed -i 's/Got to /Go to/g' fixing.html
> 
> Also the Maproulette paragraph is no longer accurate now that more than
> one challenge has been created.

Fixed. Thank you!

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Sebastiaan Couwenberg]

On 02/21/2017 05:40 PM, Jochen Topf wrote:
> Find all challenges and instructions here:
> http://area.jochentopf.com/fixing.html

My OCD complains about the typo before the challenge links, please do

 sed -i 's/Got to /Go to/g' fixing.html

Also the Maproulette paragraph is no longer accurate now that more than
one challenge has been created.

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Wed, Feb 15, 2017 at 04:20:53PM +0100, Jochen Topf wrote:
> There are a lot of (multi)polygons in OSM that are broken in one way or
> another. And we have to fix them. While some of the broken ones appear
> on the map just fine, some don't appear and some mess up the map. And
> some of those that appear fine on the main OSM map will not show up on
> other maps where different software is used.
> [...]

A week later and most of the errors I posted in those Maproulette
challenges have been fixed. Thank you!

You can see the error numbers trending down at
http://area.jochentopf.com/stats/#intersections

But there is much more to do... I posted two new challenges, one with
landuse polygons with self-intersections very similar to the challenges
posted last week and mostly easy to fix. The other contains "open
rings", multipolygon relations where the polygon boundary doesn't form a
closed ring. Some of them are a bit trickier to fix.

Find all challenges and instructions here:
http://area.jochentopf.com/fixing.html

Keep up the good work!

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Tomas Straupis]

2017-02-18 16:26 GMT+02:00 Dave F wrote:
> Why do you believe this to be a problem? It may be pointless, but not an
> error.

  As I pointed out there is no problem with geometry. So yes, it is
not an error.
  But import(?) with lots of errors in ETL or source data.

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Dave F]

Hi

Why do you believe this to be a problem? It may be pointless, but not an 
error.


I need to point out none of these given examples are multi-polygons.

DaveF

On 18/02/2017 11:37, Tomas Straupis wrote:

   Another interesting example are polygons like this:
   http://www.openstreetmap.org/way/400182030

   Polygon geometry is fine here, but it has one pointless node close
to one of the nodes. Pointless because it does not influence the final
geometry of the polygon.
   And if you look around there are a lot of such buildings with
pointless nodes. I would say almost every building has them...




---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Maarten Deen]

On 2017-02-18 12:37, Tomas Straupis wrote:

Another interesting example are polygons like this:
  http://www.openstreetmap.org/way/400182030

  Polygon geometry is fine here, but it has one pointless node close
to one of the nodes. Pointless because it does not influence the final
geometry of the polygon.
  And if you look around there are a lot of such buildings with
pointless nodes. I would say almost every building has them...


I suppose they have the same origin as the nodes outside the main 
building causing the intersecting buildings that we fix in this 
maproulette: a click to many and there is another node.


Maarten

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Tomas Straupis]

  Another interesting example are polygons like this:
  http://www.openstreetmap.org/way/400182030

  Polygon geometry is fine here, but it has one pointless node close
to one of the nodes. Pointless because it does not influence the final
geometry of the polygon.
  And if you look around there are a lot of such buildings with
pointless nodes. I would say almost every building has them...

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Florian Lohoff]

On Sat, Feb 18, 2017 at 10:26:42AM +, Andy Townsend wrote:
> On 18/02/2017 10:15, Florian Lohoff wrote:
> >
> >After Europe was fixed in <24h it seems i started to wade through Africa
> >and i am seeing some error class which makes me wonder is somebody did
> >some large scale autonomous Aerial scan for buildings.
> >
> >There are literally hundrets of building which have 4 edges as nodes
> >but them beeing connected over cross so that a construct like a
> >butterfly resembles.
> 
> Any chance of a link to an example?

https://www.openstreetmap.org/way/448649947/history
https://www.openstreetmap.org/way/425323610/history
https://www.openstreetmap.org/way/425323608/history

I have definitly fixed more than 50 of these - i skipped the above
3 now - still they are in the maproulette challenge 

Users/Mappers are different ones - most carry HOT or MissingMap hashtags
still i wonder how people produce these ...

Flo
-- 
Florian Lohoff f...@zz.de
 UTF-8 Test: The  ran after a , but the  ran away


signature.asc
Description: Digital signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Sat, Feb 18, 2017 at 01:10:13PM +0200, Tomas Straupis wrote:
> >> There are literally hundrets of building which have 4 edges as nodes
> >> but them beeing connected over cross so that a construct like a
> >> butterfly resembles.
> >
> > Any chance of a link to an example?
> 
>   I guess Florian ment geometries like this:
>   http://www.openstreetmap.org/way/460032394
> 
>   There are indeed plenty of these in "less populated" areas,
> especially areas with round buildings...

Probably the result of some HOT mapping thing done by inexperiences
mappers...

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Tomas Straupis]

>> There are literally hundrets of building which have 4 edges as nodes
>> but them beeing connected over cross so that a construct like a
>> butterfly resembles.
>
> Any chance of a link to an example?

  I guess Florian ment geometries like this:
  http://www.openstreetmap.org/way/460032394

  There are indeed plenty of these in "less populated" areas,
especially areas with round buildings...

-- 
Tomas

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Andy Townsend]

On 18/02/2017 10:15, Florian Lohoff wrote:


After Europe was fixed in <24h it seems i started to wade through Africa
and i am seeing some error class which makes me wonder is somebody did
some large scale autonomous Aerial scan for buildings.

There are literally hundrets of building which have 4 edges as nodes
but them beeing connected over cross so that a construct like a
butterfly resembles.


Any chance of a link to an example?

Best Regards,

Andy


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Florian Lohoff]

On Wed, Feb 15, 2017 at 04:20:53PM +0100, Jochen Topf wrote:
> Yesterday I launched several Maproulette challenges that allow you to
> easily help with the "cleaning up" effort. Read
> http://area.jochentopf.com/fixing.html for more details on those
> Maproulette challenges. This is a huge effort that will take a long time
> and we really need any help we get. The challenges posted today are only
> the beginning. They only show the about 6,500 ways worldwide tagged as
> buildings (with less than 100 nodes) where the way intersects itself.

After Europe was fixed in <24h it seems i started to wade through Africa
and i am seeing some error class which makes me wonder is somebody did
some large scale autonomous Aerial scan for buildings.

There are literally hundrets of building which have 4 edges as nodes
but them beeing connected over cross so that a construct like a
butterfly resembles.

I have never seen a human produce something like this.

One can easily spot human errors like using "extrude" and Josm leaving
the old edges in place, "duplicate" edges which make the polygon
self-intersect within 10cm etc ..

Flo
-- 
Florian Lohoff f...@zz.de
 UTF-8 Test: The  ran after a , but the  ran away


signature.asc
Description: Digital signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Oleksiy Muzalyev]

On 18.02.17 08:43, Jochen Topf wrote:

On Sat, Feb 18, 2017 at 07:50:07AM +0100, Oleksiy Muzalyev wrote:

On 15.02.17 23:51, Sarah Hoffmann wrote:

... we are
currently discussing about changing the algorithm that assembles the
polygons[1]. The new algorithms will be a lot faster but that comes
at the price that it is less tolerant with invalid geometries. A lot
of bad geometries that are currently still drawn some way or another
will be simply dropped. I'm convinced that in the long run this
stricter handling will be good not only for data consumers but also
for mappers, who will see immediately when they made a mistake.
...


Hi,
It would be a good idea to generate automatically a message to an author or
authors of this geometry asking them to fix it, explaining shortly what an
error they committed, and informing them that it would be deleted, if they
do not fix it. And if there is no reaction after a tolerance period of say
one week, then drop it automatically.

Most multipolygons we are talking about here haven't been touched in
years. So this doesn't really apply. Most of this is about cleaning up
the backlog. If you look at the stats on http://area.jochentopf.com/stats/
you'll see that the number of (multi)polygons is growing constantly, but
the number of errors is not. This tells me that it is mostly a problem with
old data.

  We could inform authors when new problems are coming up, but I suspect
that this is very difficult. For one, there can be several people
involved and you don't know which fault is was. Then there is the
question of what to tell the user. If you send them an email "Your
multipolygon id 1234567 is broken", that is not very helpful for them.
We need at a minimum some guidance on how to fix things. But this
depends at least on the editor used, the language of the user, the type
of problem and the skill level of the user. Ugh.


Deleting objects from the map without a warning may cause suspicions and
misunderstanding. For example, there are areas mapped as
boundary=protected_area or landuse=nature_reserve , but local fishermen who
want to continue fishing in the area may not like it, or at least have
doubts about its shape. By deleting a Nature Reserve from the map the script
may inadvertently interfere into a balance situation.

This is why we are having this discussion. We do not want to remove
anything from the map lightly. We are doing this only after taking any
measure we can to fix those cases. In the long run the situation will
get better, because at the moment some of those "critical" polygons you
are talking about might not show up or show up wrong on the map because
of the errors they contain that the renderer is trying to fix and doing
so in the wrong way. We are doing all this to improve the accuracy of
the map!

Jochen


OK. Then I agree. Indeed sometimes it is easier to map from scratch than 
to clean up speaking figuratively the Augean Stables.


After all, there are Wiki pages and Youtube videos which explain how to 
create multipoligons correctly.


Oleksiy


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Sat, Feb 18, 2017 at 07:50:07AM +0100, Oleksiy Muzalyev wrote:
> On 15.02.17 23:51, Sarah Hoffmann wrote:
> > ... we are
> > currently discussing about changing the algorithm that assembles the
> > polygons[1]. The new algorithms will be a lot faster but that comes
> > at the price that it is less tolerant with invalid geometries. A lot
> > of bad geometries that are currently still drawn some way or another
> > will be simply dropped. I'm convinced that in the long run this
> > stricter handling will be good not only for data consumers but also
> > for mappers, who will see immediately when they made a mistake.
> > ...
> > 
> Hi,
> It would be a good idea to generate automatically a message to an author or
> authors of this geometry asking them to fix it, explaining shortly what an
> error they committed, and informing them that it would be deleted, if they
> do not fix it. And if there is no reaction after a tolerance period of say
> one week, then drop it automatically.

Most multipolygons we are talking about here haven't been touched in
years. So this doesn't really apply. Most of this is about cleaning up
the backlog. If you look at the stats on http://area.jochentopf.com/stats/
you'll see that the number of (multi)polygons is growing constantly, but
the number of errors is not. This tells me that it is mostly a problem with
old data.

 We could inform authors when new problems are coming up, but I suspect
that this is very difficult. For one, there can be several people
involved and you don't know which fault is was. Then there is the
question of what to tell the user. If you send them an email "Your
multipolygon id 1234567 is broken", that is not very helpful for them.
We need at a minimum some guidance on how to fix things. But this
depends at least on the editor used, the language of the user, the type
of problem and the skill level of the user. Ugh.

> Deleting objects from the map without a warning may cause suspicions and
> misunderstanding. For example, there are areas mapped as
> boundary=protected_area or landuse=nature_reserve , but local fishermen who
> want to continue fishing in the area may not like it, or at least have
> doubts about its shape. By deleting a Nature Reserve from the map the script
> may inadvertently interfere into a balance situation.

This is why we are having this discussion. We do not want to remove
anything from the map lightly. We are doing this only after taking any
measure we can to fix those cases. In the long run the situation will
get better, because at the moment some of those "critical" polygons you
are talking about might not show up or show up wrong on the map because
of the errors they contain that the renderer is trying to fix and doing
so in the wrong way. We are doing all this to improve the accuracy of
the map!

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Oleksiy Muzalyev]

On 15.02.17 23:51, Sarah Hoffmann wrote:

... we are
currently discussing about changing the algorithm that assembles the
polygons[1]. The new algorithms will be a lot faster but that comes
at the price that it is less tolerant with invalid geometries. A lot
of bad geometries that are currently still drawn some way or another
will be simply dropped. I'm convinced that in the long run this
stricter handling will be good not only for data consumers but also
for mappers, who will see immediately when they made a mistake.
...


Hi,
It would be a good idea to generate automatically a message to an author 
or authors of this geometry asking them to fix it, explaining shortly 
what an error they committed, and informing them that it would be 
deleted, if they do not fix it. And if there is no reaction after a 
tolerance period of say one week, then drop it automatically.


Deleting objects from the map without a warning may cause suspicions and 
misunderstanding. For example, there are areas mapped as 
boundary=protected_area or landuse=nature_reserve , but local fishermen 
who want to continue fishing in the area may not like it, or at least 
have doubts about its shape. By deleting a Nature Reserve from the map 
the script may inadvertently interfere into a balance situation.


Best regards,
Oleksiy

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Christoph Hormann]

On Friday 17 February 2017, Jochen Topf wrote:
>
> Lets not get this thread hijacked by theoretical ideas about how to
> detect wooded areas. This thread is about broken multipolygons.

I did not mean to hijack this thread, i meant to provide context for the 
decision on if to keep the imported wood/forest polygons.

> I'd rather have more specific challenges addressing
> exactly one problem, for instance "Broken multipolygons of certain
> types of landcover data imported from Corinne in Sweden". That is
> something I can extract from OSM data and that I can explain to
> people how to fix after consulting with local mappers on how best to
> do this.

In principle that is a good idea, for the Scandinavian wood/forest 
polygons however the problem would be that likely for a large fraction 
of the mappers willing and able to work on this the best solution will 
often be to delete the polygon in question.  This should of course not 
happen against the desires of the local community.  This problem is the 
main reason why i brought up the subject here.

In terms of specific challenges that might be useful - waterbodies is an 
obvious candidate.  There does not appear to be any specific 
concentration of broken waterbody polygons related to an import, most 
of these are genuine manual mapping errors.  Although many of these are 
also substandard in terms of mapping quality very few are so bad it 
would be better to delete them than to keep them.

Of course this task would be technically non-trivial, many of these 
polygons are large, a lot of big river polygons that should be split 
into smaller ones etc.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Jochen Topf]

On Fri, Feb 17, 2017 at 11:11:12AM +0100, Christoph Hormann wrote:
> On Friday 17 February 2017, Andreas Vilén wrote:
> >
> > We are aware of the Corine import problems and have discussed them
> > locally at least in Sweden. Our community is very loose with not much
> > activity in mailing lists or other media, but so far consensus has
> > been not to remove Corine if it's not replaced by improved data.
> >
> > I have done some cleanup myself mostly around Kalmar, but in the huge
> > Northern parts of the country it seems unmanagable. Some users have
> > made good progress in the Bergslagen area though.
> 
> Well - the question you should probably ask yourselves is if this data 
> is of any help when you map in these areas.  I find it doubtful that it 
> is and areas like here:
> 
> http://www.openstreetmap.org/#map=13/56.7935/16.0168
> 
> support this impression.  IMO it would be a good idea to concentrate on 
> what you gain and not too much look at what you loose.
> 
> If there are worries in the local community about how to efficiently map 
> large wooded areas there are other methods that would be much better 
> suited.  Forests can be positively detected on multispectral imagery 
> with good reliability - in contrast to land cover classification data 
> sets like Corine which essentially only specify the least unlikely of a 
> fixed set of classes.  Producing a conservative data set this way (i.e. 
> one that only includes area which are clearly wooded), splitting this 
> into reasonably small chunks and providing this to mappers to avoid the 
> need for a lot of large scale tracing work seems a much more productive 
> way and much more compatible with normal manual mapping in OSM.

Lets not get this thread hijacked by theoretical ideas about how to
detect wooded areas. This thread is about broken multipolygons.

The issue at hand here is that there are a lot of broken multipolygons
out there. Some are probably from Corinne data. Now there are these
options:

a) Do nothing. Broken MPs will disappear once osm2pgsql switches.
b) Remove existing MPs and start from scratch.
c) Repair existing MPs.

From some of the posts in this thread, c) doesn't seem to be a good
option. Both a) and b) will result in those MPs disappearing from the
map first, before things get better. In the case of a) they will all
disappear one day, but the broken data is still there. In the case of b)
we can go through them and replace them by better data piece by piece.

I am willing to talk with the Swedish community (or any other) about
how best to approach this. I can generate special Maproulette challenges
for specific areas, in fact I think this is a better way then having
generic challenges for the whole world. If you work on such a challenge,
you might be thrown from an error in Borneo to one in Sweden to one in
Antarctica. And the problems are different everywhere. I'd rather have
more specific challenges addressing exactly one problem, for instance
"Broken multipolygons of certain types of landcover data imported from
Corinne in Sweden". That is something I can extract from OSM data and
that I can explain to people how to fix after consulting with local
mappers on how best to do this.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Paul Norman]

On 2/16/2017 2:51 AM, Yves wrote:
Is there an example where the community has cleaned up the Corine 
multi polygons instead of starting from scratch? 


I've done some Corine cleanup, mainly consisting of deleting obviously 
wrong data. Where I've remapped it's been faster to delete it and start 
from scratch.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Christoph Hormann]

On Friday 17 February 2017, Andreas Vilén wrote:
>
> We are aware of the Corine import problems and have discussed them
> locally at least in Sweden. Our community is very loose with not much
> activity in mailing lists or other media, but so far consensus has
> been not to remove Corine if it's not replaced by improved data.
>
> I have done some cleanup myself mostly around Kalmar, but in the huge
> Northern parts of the country it seems unmanagable. Some users have
> made good progress in the Bergslagen area though.

Well - the question you should probably ask yourselves is if this data 
is of any help when you map in these areas.  I find it doubtful that it 
is and areas like here:

http://www.openstreetmap.org/#map=13/56.7935/16.0168

support this impression.  IMO it would be a good idea to concentrate on 
what you gain and not too much look at what you loose.

If there are worries in the local community about how to efficiently map 
large wooded areas there are other methods that would be much better 
suited.  Forests can be positively detected on multispectral imagery 
with good reliability - in contrast to land cover classification data 
sets like Corine which essentially only specify the least unlikely of a 
fixed set of classes.  Producing a conservative data set this way (i.e. 
one that only includes area which are clearly wooded), splitting this 
into reasonably small chunks and providing this to mappers to avoid the 
need for a lot of large scale tracing work seems a much more productive 
way and much more compatible with normal manual mapping in OSM.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Andreas Vilén]

Hi there,

We are aware of the Corine import problems and have discussed them locally at 
least in Sweden. Our community is very loose with not much activity in mailing 
lists or other media, but so far consensus has been not to remove Corine if 
it's not replaced by improved data.

I have done some cleanup myself mostly around Kalmar, but in the huge Northern 
parts of the country it seems unmanagable. Some users have made good progress 
in the Bergslagen area though.

/Andreas

Skickat från min iPhone

> 16 feb. 2017 kl. 10:53 skrev Christoph Hormann :
> 
>> On Wednesday 15 February 2017, Sarah Hoffmann wrote:
>> 
>> There are some notable holes, for example in the woods of
>> Scandinavia. It would be great if they are gone by the time we
>> switch the software.
> 
> A general recommendation here to the Scandinavian communities - you 
> might seriously want to consider removing the forest/wood polygons from 
> the old Corine landcover imports.  These are:
> 
> * factually very inaccurate, usually indicating this area is mostly tree 
> covered rather than this area is actually forested.
> * not aligned to other geometries, in particular water bodies.
> * non-maintainable due to the polygons size leading to mappers widely 
> drawing other stuff over it.
> 
> Examples:
> 
> http://mc.bbbike.org/mc/?lon=8.836311=58.848538=14=3=google-satellite=mapnik=mapbox-satellite
> http://mc.bbbike.org/mc/?lon=27.615479=60.772083=13=3=google-satellite=mapnik=mapbox-satellite
> http://mc.bbbike.org/mc/?lon=26.339005=61.247215=13=3=google-satellite=mapnik=mapbox-satellite
> 
> Newly mapping wood areas from scratch here would likely lead to better 
> and better maintainable data in the long terms.  You could even 
> consider importing newer high quality data with a more sensible polygon 
> size.
> 
> -- 
> Christoph Hormann
> http://www.imagico.de/
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Martin Koppenhoefer]

2017-02-16 11:51 GMT+01:00 Yves :

> Is there an example where the community has cleaned up the Corine multi
> polygons instead of starting from scratch?



I have spent some time tidying up some pieces of imported Corine geometry
in the past, but I could reuse literally nothing of the imported nodes,
they were all off when looking at it at a high zoom (not surprising, as
Corine was never intended for a usecase like OSM, it is a medium zoom level
dataset).

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Yves]

Is there an example where the community has cleaned up the Corine multi 
polygons instead of starting from scratch? 

Le 16 février 2017 10:53:42 GMT+01:00, Christoph Hormann  
a écrit :
>On Wednesday 15 February 2017, Sarah Hoffmann wrote:
>>
>> There are some notable holes, for example in the woods of
>> Scandinavia. It would be great if they are gone by the time we
>> switch the software.
>
>A general recommendation here to the Scandinavian communities - you 
>might seriously want to consider removing the forest/wood polygons from
>
>the old Corine landcover imports.  These are:
>
>* factually very inaccurate, usually indicating this area is mostly
>tree 
>covered rather than this area is actually forested.
>* not aligned to other geometries, in particular water bodies.
>* non-maintainable due to the polygons size leading to mappers widely 
>drawing other stuff over it.
>
>Examples:
>
>http://mc.bbbike.org/mc/?lon=8.836311=58.848538=14=3=google-satellite=mapnik=mapbox-satellite
>http://mc.bbbike.org/mc/?lon=27.615479=60.772083=13=3=google-satellite=mapnik=mapbox-satellite
>http://mc.bbbike.org/mc/?lon=26.339005=61.247215=13=3=google-satellite=mapnik=mapbox-satellite
>
>Newly mapping wood areas from scratch here would likely lead to better 
>and better maintainable data in the long terms.  You could even 
>consider importing newer high quality data with a more sensible polygon
>
>size.
>
>-- 
>Christoph Hormann
>http://www.imagico.de/
>
>___
>talk mailing list
>talk@openstreetmap.org
>https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Christoph Hormann]

On Wednesday 15 February 2017, Sarah Hoffmann wrote:
>
> There are some notable holes, for example in the woods of
> Scandinavia. It would be great if they are gone by the time we
> switch the software.

A general recommendation here to the Scandinavian communities - you 
might seriously want to consider removing the forest/wood polygons from 
the old Corine landcover imports.  These are:

* factually very inaccurate, usually indicating this area is mostly tree 
covered rather than this area is actually forested.
* not aligned to other geometries, in particular water bodies.
* non-maintainable due to the polygons size leading to mappers widely 
drawing other stuff over it.

Examples:

http://mc.bbbike.org/mc/?lon=8.836311=58.848538=14=3=google-satellite=mapnik=mapbox-satellite
http://mc.bbbike.org/mc/?lon=27.615479=60.772083=13=3=google-satellite=mapnik=mapbox-satellite
http://mc.bbbike.org/mc/?lon=26.339005=61.247215=13=3=google-satellite=mapnik=mapbox-satellite

Newly mapping wood areas from scratch here would likely lead to better 
and better maintainable data in the long terms.  You could even 
consider importing newer high quality data with a more sensible polygon 
size.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Warin]

On 16-Feb-17 10:52 AM, Paul Norman wrote:

On 2/15/2017 2:51 PM, Sarah Hoffmann wrote:

There is a comparison map where you can see the changes:

https://osmium.osm2pgsql.paulnorman.ca

There are some notable holes, for example in the woods of
Scandinavia. It would be great if they are gone by the time we
switch the software.


Just to note, rendering is currently disabled as I'm reloading the 
database with the latest code and doing some other tests.


Arr .. I was puzzled. Thanks.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Paul Norman]

On 2/15/2017 2:51 PM, Sarah Hoffmann wrote:

There is a comparison map where you can see the changes:

https://osmium.osm2pgsql.paulnorman.ca

There are some notable holes, for example in the woods of
Scandinavia. It would be great if they are gone by the time we
switch the software.


Just to note, rendering is currently disabled as I'm reloading the 
database with the latest code and doing some other tests.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Walter Nordmann]

Hi sarah,

i'm doing the missing boundaries task trying to fix as many bad boundary 
polygons as possible.


Working with osm2pgsql too i decided some month ago to disallow 
osm2pgsql  to automatically fix these geometries by adding 
--exclude-invalid-polygon. In the first days i got a lot of missings but 
the commiunity helped me to fix all of them :)


go on and disable as much of automatic "fixing" as possible. sooner or 
later these real errors - which are unvisible to end users - will be fixed.


see you in karlsruhe, walter


Am 15.02.2017 um 23:51 schrieb Sarah Hoffmann:

Hi,

let me add a bit of motivation to this. For osm2pgsql, the software
that process the OSM data for rendering the map on osm.org, we are
currently discussing about changing the algorithm that assembles the
polygons[1]. The new algorithms will be a lot faster but that comes
at the price that it is less tolerant with invalid geometries. A lot
of bad geometries that are currently still drawn some way or another
will be simply dropped. I'm convinced that in the long run this
stricter handling will be good not only for data consumers but also
for mappers, who will see immediately when they made a mistake.
However, in the short run a switch from the old algorithm to the
new one will leave a few bald patches on the map.

There is a comparison map where you can see the changes:

https://osmium.osm2pgsql.paulnorman.ca

There are some notable holes, for example in the woods of
Scandinavia. It would be great if they are gone by the time we
switch the software.

Sarah


[1] https://github.com/openstreetmap/osm2pgsql/pull/684

On Wed, Feb 15, 2017 at 04:20:53PM +0100, Jochen Topf wrote:

There are a lot of (multi)polygons in OSM that are broken in one way or
another. And we have to fix them. While some of the broken ones appear
on the map just fine, some don't appear and some mess up the map. And
some of those that appear fine on the main OSM map will not show up on
other maps where different software is used.

A while ago I set up a web page at http://area.jochentopf.com/ and a
Github repository at https://github.com/osmlab/fixing-polygons-in-osm
devoted to that issue that I never announced properly. Go there and read
up in more detail where the problems are and how we are going to fix
them.

Yesterday I launched several Maproulette challenges that allow you to
easily help with the "cleaning up" effort. Read
http://area.jochentopf.com/fixing.html for more details on those
Maproulette challenges. This is a huge effort that will take a long time
and we really need any help we get. The challenges posted today are only
the beginning. They only show the about 6,500 ways worldwide tagged as
buildings (with less than 100 nodes) where the way intersects itself.

I have decided to start with a simple problem like this, to learn how
the Maproulette stuff works and how well, you, the community, responds.
Especially for beginners fixing those building is much easier than
starting with 10,000 node multipolygons with possibly multiple errors in
them. (If you want to, you can still do that. All multipolygon errors
show up in the OSM Inspector areas view at
http://tools.geofabrik.de/osmi/?view=areas )

Jochen
--
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons

[Sarah Hoffmann]

Hi,

let me add a bit of motivation to this. For osm2pgsql, the software
that process the OSM data for rendering the map on osm.org, we are
currently discussing about changing the algorithm that assembles the
polygons[1]. The new algorithms will be a lot faster but that comes
at the price that it is less tolerant with invalid geometries. A lot
of bad geometries that are currently still drawn some way or another
will be simply dropped. I'm convinced that in the long run this
stricter handling will be good not only for data consumers but also
for mappers, who will see immediately when they made a mistake.
However, in the short run a switch from the old algorithm to the
new one will leave a few bald patches on the map.

There is a comparison map where you can see the changes:

https://osmium.osm2pgsql.paulnorman.ca

There are some notable holes, for example in the woods of
Scandinavia. It would be great if they are gone by the time we
switch the software.

Sarah


[1] https://github.com/openstreetmap/osm2pgsql/pull/684

On Wed, Feb 15, 2017 at 04:20:53PM +0100, Jochen Topf wrote:
> There are a lot of (multi)polygons in OSM that are broken in one way or
> another. And we have to fix them. While some of the broken ones appear
> on the map just fine, some don't appear and some mess up the map. And
> some of those that appear fine on the main OSM map will not show up on
> other maps where different software is used.
> 
> A while ago I set up a web page at http://area.jochentopf.com/ and a
> Github repository at https://github.com/osmlab/fixing-polygons-in-osm
> devoted to that issue that I never announced properly. Go there and read
> up in more detail where the problems are and how we are going to fix
> them.
> 
> Yesterday I launched several Maproulette challenges that allow you to
> easily help with the "cleaning up" effort. Read
> http://area.jochentopf.com/fixing.html for more details on those
> Maproulette challenges. This is a huge effort that will take a long time
> and we really need any help we get. The challenges posted today are only
> the beginning. They only show the about 6,500 ways worldwide tagged as
> buildings (with less than 100 nodes) where the way intersects itself.
> 
> I have decided to start with a simple problem like this, to learn how
> the Maproulette stuff works and how well, you, the community, responds.
> Especially for beginners fixing those building is much easier than
> starting with 10,000 node multipolygons with possibly multiple errors in
> them. (If you want to, you can still do that. All multipolygon errors
> show up in the OSM Inspector areas view at
> http://tools.geofabrik.de/osmi/?view=areas )
> 
> Jochen
> -- 
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk