Re: [OSM-talk] Fixing broken multipolygons, some notes

[Christoph Hormann]

On Monday 20 March 2017, Andreas Vilén wrote:
>
> It's not up to me to decide if this data is to be deleted or not. If
> you want to do that, raise the question with each respective
> country's mailing list.

I don't want to push the issue - and there is little chance for such a 
suggestion without significant parts of the local community feeling a 
need for this on their own.

Since I am pretty sure quite a few of the mappers from Sweden and 
Finland read this list people are likely aware of the problem now 
(likely were before in some way as well).

Despite this the whole matter also has a more global component of 
course.  Mapping of the large wooded areas of the planet is a problem 
in OSM that is going to bother us for the forseeable future.  Right now 
the only areas where you can produce larger area maps with forest 
rendering in good quality based on OSM data alone are western and 
central Europe (mostly, there are gaps even in that, in particular in 
the Italian Alps).  And these are obviously not particularly difficult 
in terms of forest mapping.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Jochen Topf]

On Mon, Mar 20, 2017 at 07:15:47AM +0100, Jochen Topf wrote:
> On Mon, Mar 20, 2017 at 06:27:05AM +0100, Andreas Vilén wrote:
> > It's not up to me to decide if this data is to be deleted or not. If
> > you want to do that, raise the question with each respective country's
> > mailing list.
> 
> I was under the impression that were in contact with the Swedish
 ^you

> community on this topic and that the Swedish community had decided they
> wanted to keep the data. So I thought this issue was settled.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Jochen Topf]

On Mon, Mar 20, 2017 at 06:27:05AM +0100, Andreas Vilén wrote:
> It's not up to me to decide if this data is to be deleted or not. If
> you want to do that, raise the question with each respective country's
> mailing list.

I was under the impression that were in contact with the Swedish
community on this topic and that the Swedish community had decided they
wanted to keep the data. So I thought this issue was settled.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Andreas Vilén]

There is a difference between data errors that the osmi will detect and bad 
quality map data. Corine is the latter.

It's not up to me to decide if this data is to be deleted or not. If you want 
to do that, raise the question with each respective country's mailing list.

/Andreas

Skickat från min iPhone

> 19 mars 2017 kl. 19:32 skrev Christoph Hormann :
> 
>> On Sunday 19 March 2017, Andreas Vilén wrote:
>> 
>> Also, as has been pointed out earlier, Corine data might be bad, but
>> does not contain that many pure data errors as we define them.
> 
> That is not quite accurate in my experience.  As i explained in
> 
> https://lists.openstreetmap.org/pipermail/talk/2017-February/077553.html
> 
> landcover mappings like corine are based on selecting the least unlikely 
> of a fixed set of landcover classes at a certain scale based on certain 
> criteria and reference areas and this frequently produces completely 
> bogus results.
> 
> In areas like here:
> 
> http://www.openstreetmap.org/#map=13/56.7935/16.0168
> http://www.openstreetmap.org/#map=13/62.7015/25.6678
> 
> the Corine based landcover data in my eyes has not connection to reality 
> at all, it is factually simply wrong.
> 
> I understand that the perspective to loose all the data and to be 
> without any forest mapping until manual mapping has filled the gaps 
> seems not so pleasant but i am pretty sure there are a lot of people 
> from abroad who would be glad to help you with either manual forest 
> mapping or importing better quality data in a way that is better 
> maintainable and more compatible with manual mapping activity.
> 
> However as long as the bad quality Corine data is there and meant to 
> stay few people are interested in editing this mostly meaningless data.
> 
> -- 
> Christoph Hormann
> http://www.imagico.de/
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Christoph Hormann]

On Sunday 19 March 2017, Andreas Vilén wrote:
>
> Also, as has been pointed out earlier, Corine data might be bad, but
> does not contain that many pure data errors as we define them.

That is not quite accurate in my experience.  As i explained in

https://lists.openstreetmap.org/pipermail/talk/2017-February/077553.html

landcover mappings like corine are based on selecting the least unlikely 
of a fixed set of landcover classes at a certain scale based on certain 
criteria and reference areas and this frequently produces completely 
bogus results.

In areas like here:

http://www.openstreetmap.org/#map=13/56.7935/16.0168
http://www.openstreetmap.org/#map=13/62.7015/25.6678

the Corine based landcover data in my eyes has not connection to reality 
at all, it is factually simply wrong.

I understand that the perspective to loose all the data and to be 
without any forest mapping until manual mapping has filled the gaps 
seems not so pleasant but i am pretty sure there are a lot of people 
from abroad who would be glad to help you with either manual forest 
mapping or importing better quality data in a way that is better 
maintainable and more compatible with manual mapping activity.

However as long as the bad quality Corine data is there and meant to 
stay few people are interested in editing this mostly meaningless data.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Mattias Dalkvist]

On Sun, Mar 19, 2017 at 12:09 PM, Martin Koppenhoefer <
dieterdre...@gmail.com> wrote:

>
> On 18 Mar 2017, at 21:40, Sandor Seres  wrote:
>
> In another style, typical land related names are on the water like here
> http://osm.org/go/0Tt1PZIt-?layers=T .
>
>
> seems like either a bad import or localities on the sea, e.g. here:
> http://www.openstreetmap.org/node/2535180885
> 
>
>

The names in that case are bays and islands but it looks like there are 2
lake relations and only one of them have the islands as inner members
http://www.openstreetmap.org/relation/175607#map=14/60.4445/10.2434
http://www.openstreetmap.org/relation/2922000#map=14/60.4435/10.2359

-- 
Dalkvist
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[James]

Also if I remember correctly the whole point was to optimize/be able to
process osm data faster by not having to deal with so many errors(each case
can slow down the processing)

On Mar 19, 2017 7:23 AM, "Martin Koppenhoefer" 
wrote:

>
>
> sent from a phone
>
> > On 19 Mar 2017, at 10:03, Andreas Vilén  wrote:
> >
> > The main problem with Corine is that oftentimes the landuse data
> overlaps villages (which I found when I mapped mountain villages in
> southern Spain last week as well)
>
>
> whenever looking closeup at any corine data it was hardly possible to see
> the same borders they have in the aerial imagery. Often they resemble, but
> never (in the cases I looked at) the border was drawn where I'd expect a
> human mapper to draw it. The data doesn't seem suitable for small scale
> maps (small scale is what you typically observe on the ground)
>
> Cheers,
> Martin
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Martin Koppenhoefer]

sent from a phone

> On 19 Mar 2017, at 10:03, Andreas Vilén  wrote:
> 
> The main problem with Corine is that oftentimes the landuse data overlaps 
> villages (which I found when I mapped mountain villages in southern Spain 
> last week as well)


whenever looking closeup at any corine data it was hardly possible to see the 
same borders they have in the aerial imagery. Often they resemble, but never 
(in the cases I looked at) the border was drawn where I'd expect a human mapper 
to draw it. The data doesn't seem suitable for small scale maps (small scale is 
what you typically observe on the ground)

Cheers,
Martin 
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Martin Koppenhoefer]

sent from a phone

> On 18 Mar 2017, at 21:40, Sandor Seres  wrote:
> 
> In another style, typical land related names are on the water like here 
> http://osm.org/go/0Tt1PZIt-?layers=T .


seems like either a bad import or localities on the sea, e.g. here: 
http://www.openstreetmap.org/node/2535180885

Cheers,
Martin ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Andreas Vilén]

To be fair, "the local community" in this case is probably mostly me and my
interpretation of the Swedish community's reaction when someone tried to
remove Corine imagery rather carelessly (introduced a lot of new errors) a
couple of years ago. It's too bad that we're talking about a region when no
one from that region is posting in this mailing list (even though I'm sure
many read it).

Also, as has been pointed out earlier, Corine data might be bad, but does
not contain that many pure data errors as we define them. This is
approximately the area in Sandor's link in OSM inspector:
http://tools.geofabrik.de/osmi/?view=areas=10.35829=60.44245=10
This is probably the worst problem area but that has been mapped by hand:
http://tools.geofabrik.de/osmi/?view=areas=14.63086=56.93988=9

The main problem with Corine is that oftentimes the landuse data overlaps
villages (which I found when I mapped mountain villages in southern Spain
last week as well) and also when someone has been mapping by hand and
doesn't clean up the Corine data while doing that (I have been doing a lot
of cleanup here, but lots of fixing remains, but as you can see, no mapping
errors:
http://tools.geofabrik.de/osmi/?view=areas=16.63835=56.73536=12
).

Maybe new error tools can be developed to find for example when Corine data
overlaps other landuse data, or it can be taught to analyze imagery for
buildings inside farmland/forest tags?

/Andreas

On Sat, Mar 18, 2017 at 11:24 PM, Frederik Ramm  wrote:

> Sandor,
>
>if I understand you correctly your basic message is "let's try and
> improve osm2pgsql's polygon interpretation rules instead of fixing the
> data", or at least "while we wait for the data fixing to be completed".
>
> I think this is not a good idea because, as you remark yourself, those
> working with the OSM source data directly will not profit from more
> magic in osm2pgsql. We would have more situations where someone
> processes OSM data with, say, QGIS and thinks "uh, I must have done
> something wrong, because on the OSM map there's a forest here and on my
> map there isn't".
>
> Fixing the data in OSM is the better approach. In many cases, broken
> polygons are a result of a broken import or careless editing and if
> MapRoulette prompts someone to open their editor in that location, they
> are likely to find quite a few other things worth of attention.
>
> > The “Fixing broken polygons”, especially programmatic/mass fixing, could
> > be more dangerous to all users.
>
> I wholly agree that, with perhaps a very few narrowly defined
> exceptions, nobody should make an attempt to fix broken polygons
> automatically because many things go wrong (and the automatic fix would
> not give us the advantage I mentioned above, that someone looking at the
> data in the area might find other things amiss).
>
> > Let us look at a map extract from the
> > mentioned Scandinavian forests
>
> It has been suggested to completely remove some CORINE-based landcover
> import in Scandinavia on the basis that not only there are many broken
> polygons, but also that it bears very little resemblance to the reality
> on the ground. People have said it is a nicely painted map but not much
> more. But I believe the local community said they preferred a nicely
> painted and somewhat wrong map over having to trace the forests from
> aerial imagery themselves ;)
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[Frederik Ramm]

Sandor,

   if I understand you correctly your basic message is "let's try and
improve osm2pgsql's polygon interpretation rules instead of fixing the
data", or at least "while we wait for the data fixing to be completed".

I think this is not a good idea because, as you remark yourself, those
working with the OSM source data directly will not profit from more
magic in osm2pgsql. We would have more situations where someone
processes OSM data with, say, QGIS and thinks "uh, I must have done
something wrong, because on the OSM map there's a forest here and on my
map there isn't".

Fixing the data in OSM is the better approach. In many cases, broken
polygons are a result of a broken import or careless editing and if
MapRoulette prompts someone to open their editor in that location, they
are likely to find quite a few other things worth of attention.

> The “Fixing broken polygons”, especially programmatic/mass fixing, could
> be more dangerous to all users.

I wholly agree that, with perhaps a very few narrowly defined
exceptions, nobody should make an attempt to fix broken polygons
automatically because many things go wrong (and the automatic fix would
not give us the advantage I mentioned above, that someone looking at the
data in the area might find other things amiss).

> Let us look at a map extract from the
> mentioned Scandinavian forests

It has been suggested to completely remove some CORINE-based landcover
import in Scandinavia on the basis that not only there are many broken
polygons, but also that it bears very little resemblance to the reality
on the ground. People have said it is a nicely painted map but not much
more. But I believe the local community said they preferred a nicely
painted and somewhat wrong map over having to trace the forests from
aerial imagery themselves ;)

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Fixing broken multipolygons, some notes

[john whelan]

There has been some discussion on the HOT mailing list that makes things a
bit clearer.

OSM in general has a fair number of things that have been added in a less
than ideal way.  It can be difficult to correct some things as we have
guidelines or recommended practises as opposed to hard and fast rules but
maproulette has managed to identify a number of areas where there is some
agreement about what needs to be corrected.

JOSM validation also tries to identify problem areas so I suspect fixing
the underlying data is the better long term solution rather than ensuring
all the different rendering systems are more robust.  Robustness costs
machine cycles​ as well.

Cheerio John

On 18 Mar 2017 4:43 pm, "Sandor Seres"  wrote:

> I am new to this list and therefore apologize for eventual
> misinterpretations and wrong stile. The motivation for the mail is a
> worrying mail on the local list about the purer osm2pgsql based maps and
> about the “broken polygons” fixing strategies. The mentioned white spots in
> the Scandinavian forests are just an illustration. By simply dropping
> broken polygons, empty spots will be typical for any area types and for any
> corners of the Planet.
>
> As I understand, osm2pgsql is an application doing data preparation from
> the OSM source data up to a DB used by many mapmakers for rendering. We can
> see that almost all OSM based public mapping system use this database and
> consequently repeat the same anomalies. Therefore, maybe, making the
> osm2pgsql more robust could be a better strategy. There is still a large
> potential for such strengthening. Just waiting for “do-ocracy” reparations
> is really a long-term strategy. Anyway, users starting from the source OSM
> data will not be affected by any of these strategies.
>
> The “Fixing broken polygons”, especially programmatic/mass fixing, could
> be more dangerous to all users. Just look at the many possible
> self-crossing fixing options. Loosely defined notions open for different
> interpretations and different sets of error criteria. Consequently, for the
> same object type we may have (and we do) different error classes and
> reparation tools. Besides the typical polygon interpretations as area (ESRI
> polygon redefinition) or as a closed polygonal line, we simply can’t find
> in the documentation what “outer”, “inner”, “hole” … notions actually mean.
> The interpretation (individual perception) of these notions is left to us
> and there we have a source of misunderstandings. For instance, if we assume
> that “outer” border polygons define the interior candidate points (points
> inside and on the border) and “inner” border polygons define (in the same
> way) exterior points of area than self-crossings, touching polygons,
> polygon overlaps, crossings… are not errors at all.
>
> However, my point here is still something else. The “broken multipolygon”
> (whatever that means) issue is just “the tip of the iceberg”. There is
> still remaining huge number of anomalies caused by area object relations
> from different area classes. I intentionally use the anomaly notion, as a
> moderate form for error, because many people/mapmakers may liv with them.
> But a modern GIS system and a vector layers based digital cartography
> cannot tolerate them. Let me present some arguments and illustrations. Let
> us look at a map extract from the mentioned Scandinavian forests here
> http://osm.org/go/0Tt1PZIt- . The example could be taken from any corner
> of the Planet and, as mentioned, there is huge number of similar cases. At
> the first glance, everything looks correct and nice (and it is). However,
> we see immediately that something is still wrong. The forest type symbols
> are placed directly over the water. In another style, typical land related
> names are on the water like here http://osm.org/go/0Tt1PZIt-?layers=T .
> Looking at the source data we can see that the lake in the middle is placed
> over an empty space (intentionally, not a hole) where the border of the
> lake runs slightly in and outside the forests. At the same time, we can see
> many forest areas inside the mentioned empty space overwritten with the
> lake that has no holes. Consequently, there are many missing islands in the
> lake and many missing forest areas in the extract. Note that only on that
> little extract there are more than 40 of the described anomalies. What
> more, there are many lakes with borders running in/out of forest areas
> (corridor border overlaps), having considerable parts over a forest and
> holes in forests, partly overlapping several disjunctive forest areas and
> so on, and the contrary. Extending the case to the Planet and other area
> types combinations we may feel the extent of the issue. There were attempts
> to compensate these problems in renderings like rendering the holes,
> rendering smaller over larger objects and so on. These actions generally do
> not work. Simply, they do good some places and damaging 

[OSM-talk] Fixing broken multipolygons, some notes

[Sandor Seres]

I am new to this list and therefore apologize for eventual
misinterpretations and wrong stile. The motivation for the mail is a
worrying mail on the local list about the purer osm2pgsql based maps and
about the "broken polygons" fixing strategies. The mentioned white spots in
the Scandinavian forests are just an illustration. By simply dropping broken
polygons, empty spots will be typical for any area types and for any corners
of the Planet. 

As I understand, osm2pgsql is an application doing data preparation from the
OSM source data up to a DB used by many mapmakers for rendering. We can see
that almost all OSM based public mapping system use this database and
consequently repeat the same anomalies. Therefore, maybe, making the
osm2pgsql more robust could be a better strategy. There is still a large
potential for such strengthening. Just waiting for "do-ocracy" reparations
is really a long-term strategy. Anyway, users starting from the source OSM
data will not be affected by any of these strategies.

The "Fixing broken polygons", especially programmatic/mass fixing, could be
more dangerous to all users. Just look at the many possible self-crossing
fixing options. Loosely defined notions open for different interpretations
and different sets of error criteria. Consequently, for the same object type
we may have (and we do) different error classes and reparation tools.
Besides the typical polygon interpretations as area (ESRI polygon
redefinition) or as a closed polygonal line, we simply can't find in the
documentation what "outer", "inner", "hole" . notions actually mean. The
interpretation (individual perception) of these notions is left to us and
there we have a source of misunderstandings. For instance, if we assume that
"outer" border polygons define the interior candidate points (points inside
and on the border) and "inner" border polygons define (in the same way)
exterior points of area than self-crossings, touching polygons, polygon
overlaps, crossings. are not errors at all. 

However, my point here is still something else. The "broken multipolygon"
(whatever that means) issue is just "the tip of the iceberg". There is still
remaining huge number of anomalies caused by area object relations from
different area classes. I intentionally use the anomaly notion, as a
moderate form for error, because many people/mapmakers may liv with them.
But a modern GIS system and a vector layers based digital cartography cannot
tolerate them. Let me present some arguments and illustrations. Let us look
at a map extract from the mentioned Scandinavian forests here
http://osm.org/go/0Tt1PZIt- . The example could be taken from any corner of
the Planet and, as mentioned, there is huge number of similar cases. At the
first glance, everything looks correct and nice (and it is). However, we see
immediately that something is still wrong. The forest type symbols are
placed directly over the water. In another style, typical land related names
are on the water like here http://osm.org/go/0Tt1PZIt-?layers=T . Looking at
the source data we can see that the lake in the middle is placed over an
empty space (intentionally, not a hole) where the border of the lake runs
slightly in and outside the forests. At the same time, we can see many
forest areas inside the mentioned empty space overwritten with the lake that
has no holes. Consequently, there are many missing islands in the lake and
many missing forest areas in the extract. Note that only on that little
extract there are more than 40 of the described anomalies. What more, there
are many lakes with borders running in/out of forest areas (corridor border
overlaps), having considerable parts over a forest and holes in forests,
partly overlapping several disjunctive forest areas and so on, and the
contrary. Extending the case to the Planet and other area types combinations
we may feel the extent of the issue. There were attempts to compensate these
problems in renderings like rendering the holes, rendering smaller over
larger objects and so on. These actions generally do not work. Simply, they
do good some places and damaging at other places. So, the question is
whether and what can we do with the problem. Just waiting for do-ocracy
based reparations is, obviously, irrational. Fortunately, the source data
has a large potential to remove most of the mentioned anomalies. Let me
present some hints in bullets for the forests, lakes and river combinations.

Assume {F0} is a set of all forest outer border polygons (closed polygonal
lines) and {F1,L0,R0} is a set of all inner forest, outer lake and outer
river border polygons (the orientations and the relations are irrelevant).
Then, you can prove the existence of minimal disjunctive simple area
coverage of the forests. In other words, you can find a set of isolated
simple areas (one outer and zero or any number of inner polygons) where any
area point is on/inside of at least one element in {F0} and never on/ inside
of any element in