Re: [Talk-ca] Multipolygon problems

[Stewart C. Russell]

On 2017-07-01 05:22 PM, Jochen Topf wrote:
> 
> There is nothing specific about woods here.

It seems, though, that the root problem in Canada is *mostly* related to
woods imported from Canvec. And we have some very large forestry
relations indeed. I was finding that the JOSM process was using 13 GB of
RAM just to load one.

> Of course you should still check all cases against sat images.

Outside cities, sat images are extremely poor in Canada. You might get
10 year old 7½-metre greyscale images. Which if you're looking at a
logging/bark beetle area could be way off current reality.

 Stewart

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Multipolygon problems

[James]

Depends where you get the data I think, canvec from ftp is different from
canvec from toporama/atlas

On Jul 1, 2017 5:44 PM, "Stewart C. Russell"  wrote:

> Hi James,
>
> > Canvec 10.0 doesnt have the issues of double tagging, just overlapping
>
> I've found a whole bunch of Canvec 10 data with this problem west of
> Sudbury. I think it may still be an issue with later versions.
>
>  Stewart
>
> ___
> Talk-ca mailing list
> Talk-ca@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-ca
>
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Multipolygon problems

[Stewart C. Russell]

Hi James,

> Canvec 10.0 doesnt have the issues of double tagging, just overlapping

I've found a whole bunch of Canvec 10 data with this problem west of
Sudbury. I think it may still be an issue with later versions.

 Stewart

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Multipolygon problems

[Jochen Topf]

On Sat, Jul 01, 2017 at 04:04:25PM -0400, Stewart C. Russell wrote:
> On 2017-07-01 04:30 AM, Frank Steggink wrote:
> > 
> > To all, this is the procedure I used yesterday, and probably something
> > similar also by Pierre.
> > * Not sure if it is a requirement, but it's better to use 64 bit Java.
> > …
> 
> Thanks for this, Frank. I think I've found a way to make this a bit
> quicker by loading a relation URL, then using your search query:
> 
> > * Eventually JOSM starts looking cluttered, because of all the extra
> > data. You can use the search query "type:way natural=wood role:outer" to
> > see if there are still rings needing work.
> 
> … then just deleting the ‘natural=wood’ from the selected ways.
> 
> I hope I'm understanding the problem correctly*: outer ways in a forest
> polygon relationship shouldn't have the ‘natural=wood’ tag? If that's
> the issue, then this should just be an auto-edit, no JOSM and
> pointy-clicky required.

There is nothing specific about woods here. An outer way in a
multipolygon relations should only have tags which apply to *that way*
specifically, not tags which apply to the whole relation. The tags on
the relation are for the polygon as a whole, the tags on the ways are
for those ways.

So if you have, say a wall surrounding a forest, you might have tags for
that wall on the ways, but the forest tags belong on the multipolygon
relation only. In most cases this simply means that there should be no
tags on the ways at all.

Of course you should still check all cases against sat images. For
instance, sometimes the whole relation can be removed and just a simple
closed way be used if there are no inner ways.

Jochen
-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

hebdoOSM Nº 362 2017-06-20-2017-06-26

[weeklyteam]

Bonjour,

Le résumé hebdomadaire n° 362 de l'actualité OpenStreetMap vient de paraître 
*en français*. Un condensé à retrouver sur :

http://www.weeklyosm.eu/fr/archives/9199/

Bonne lecture !

hebdoOSM ? 
Qui : https://wiki.openstreetmap.org/wiki/WeeklyOSM#Available_Languages 
Où : 
https://umap.openstreetmap.fr/en/map/weeklyosm-is-currently-produced-in_56718#2/8.6/108.3
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Multipolygon problems

[Stewart C. Russell]

On 2017-07-01 04:30 AM, Frank Steggink wrote:
> 
> To all, this is the procedure I used yesterday, and probably something
> similar also by Pierre.
> * Not sure if it is a requirement, but it's better to use 64 bit Java.
> …

Thanks for this, Frank. I think I've found a way to make this a bit
quicker by loading a relation URL, then using your search query:

> * Eventually JOSM starts looking cluttered, because of all the extra
> data. You can use the search query "type:way natural=wood role:outer" to
> see if there are still rings needing work.

… then just deleting the ‘natural=wood’ from the selected ways.

I hope I'm understanding the problem correctly*: outer ways in a forest
polygon relationship shouldn't have the ‘natural=wood’ tag? If that's
the issue, then this should just be an auto-edit, no JOSM and
pointy-clicky required.

cheers,
 Stewart

*: here's one of my edits, just in case I'm doing it wrong:
https://www.openstreetmap.org/changeset/49971662

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Radio-Canada - Carte Google premières nations est la meilleure - Réagissez

[Pierre Béland]

Suite aux discussions de cette semaine et à partir de la liste wikipedia, la 
Carte des établisements autochtones est complétée pour le territoire du Québec. 
Améliorations possibles bien sûr.
Following this week discussions and from the wikipedia list, the Map of 
indigenous establishment is completed for the Quebec territory. There are 
surely some possible enhancements.

Carte Overpass
http://overpass-turbo.eu/s/q6v  
Pierre 

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Multipolygon problems

[Pierre Béland]

Merci à vous deux. Pour ce qui est de Jochen, je l'invite à descendre au niveau 
des paqueretes et apprendre le respect, la communication plus positive ;)
La requête Overpass peut être utilisée directement dans JOSMFichier / 
Téléchargement depuis Overpass-API
 
Pierre 


  De : James 
 À : Frank Steggink  
Cc : Talk-CA OpenStreetMap 
 Envoyé le : samedi 1 juillet 2017 8h25
 Objet : Re: [Talk-ca] Multipolygon problems
   
Error free Québec, just in time for Canada day! Good job Pierre :-)
On Jul 1, 2017 4:34 AM, "Frank Steggink"  wrote:

Hi Jochen,

Maybe a MapRoulette challenge might even not be necessary. Yesterday I started 
to clean up a bit in Québec, but since it was already past midnight for me, I 
wanted to continue this morning. To my surprise Pierre has done a lot of work 
and now the entire province of Québec looks to be free from these errors. I 
just could find three errors, and fixed them. Bon travail, Pierre!

The OSM inspector will still be a good idea, in order to spot future errors.

To all, this is the procedure I used yesterday, and probably something similar 
also by Pierre.
* Not sure if it is a requirement, but it's better to use 64 bit Java.
* You'll need JOSM with the remote control plugin. You'll also need to start 
JOSM.
* Use Pierre's Overpass query (http://overpass-turbo.eu/s/q5 K), and zoom/pan 
to the area of interest.
* Press Run and let Overpass do its work. Don't be afraid when Overpass 
mentions that the result is huge if you have a modern computer. Last night I 
wasn't experiencing any problems with 30MB data.
* Press Export, and choose JOSM. Press "Repair query" when Overpass asks you to 
decide.
* JOSM starts downloading the data. Note that you're only getting the outer 
rings.
* Go to these rings one by one, and load data (at least you'll need the 
relationship itself).
* Remove the natural=wood tag from the outer ring.
* Eventually JOSM starts looking cluttered, because of all the extra data. You 
can use the search query "type:way natural=wood role:outer" to see if there are 
still rings needing work.
* Upload your changes. Be prepared to handle conflicts if someone else is 
working near you on this issue as well.
* After a while, check Overpass Turbo again.

I'm not sure what the update frequency is, but Pierre's changes from 4 hours 
ago were already processed.

Good luck!

Frank


On 01-07-2017 09:52, Jochen Topf wrote:

On Fri, Jun 30, 2017 at 11:47:36PM +0200, Frank Steggink wrote:

On 30-06-2017 21:21, Jochen Topf wrote:

On Fri, Jun 30, 2017 at 08:16:40PM +0200, Frank Steggink wrote:

Maybe I'm not understanding it, but in the OSM inspector [1] I just see one
case of old style multipolygon, in Manitoba. Last week, when you posted your
original message, I just saw one case in New Brunswick. IIRC, it was a park,
not even from the Canvec import.

The types of problems I am talking about don't show up in the OSM
inspector. This is not old-style multipolygons (where tags are on the
outer ways and not on the relation), but multipolygons where the tags
are on the relation AND on the ways.

Ah, ok, now I understand. Since there was a lot of discussion about old
style multipolygon tagging, and since this type of problem hasn't been added
to OSM inspector,  this wasn't immediately obvious.


In the OSM inspector other errors can be seen, but the most prevalent one is
"Touching rings". Maybe indeed a case of suboptimal mapping, but nothing
which seems urgent to me.

Here is an example of a forest multipolygon, imported by me
(canvec_fsteggink). It is still version 1, but it has tags on the relation,
not on the rings (except for the quarries): [2]
This is from Canvec v7.0. IIRC, we started at v6.0, and the last version I
know of is v10.0. Maybe v6.0 had wrong tagging, but I'm not seeing any such
cases in the OSM inspector.

So, I'd like to ask you to give a couple of examples where data imported
from Canvec is clearly wrong with regard to old style multipolygon tagging.

Here are all cases in Canada (not only those from the imports):
https://tmp.jochentopf.com/954 226a3acab882d28d8500ddef8203d/ same-tags-ca.pbf

Here is one example where you can clearly see the problem:
http://www.openstreetmap.org/r elation/541821

How difficult would it be to add this to OSM inspector? Not everybody has
Postgres running, and is able to use osm2pgsql. Yes, there is documentation,
but it requires some technical skills. Also, it would be very convenient to
have this updated daily.

It is not that difficult to add to the OSM Inspector and if I have the
time I'll work on that together with the Geofabrik people.




When we have clear examples, then it might be easier to come up with a plan
how to fix it. But so far, I see absolutely no reason why Canada stands out
in a negative way. Yes, we all acknowledge that Canvec data is suboptimal,
but as others already have pointed out, mapping 

Re: [Talk-ca] Multipolygon problems

[James]

Error free Québec, just in time for Canada day! Good job Pierre :-)

On Jul 1, 2017 4:34 AM, "Frank Steggink"  wrote:

> Hi Jochen,
>
> Maybe a MapRoulette challenge might even not be necessary. Yesterday I
> started to clean up a bit in Québec, but since it was already past midnight
> for me, I wanted to continue this morning. To my surprise Pierre has done a
> lot of work and now the entire province of Québec looks to be free from
> these errors. I just could find three errors, and fixed them. Bon travail,
> Pierre!
>
> The OSM inspector will still be a good idea, in order to spot future
> errors.
>
> To all, this is the procedure I used yesterday, and probably something
> similar also by Pierre.
> * Not sure if it is a requirement, but it's better to use 64 bit Java.
> * You'll need JOSM with the remote control plugin. You'll also need to
> start JOSM.
> * Use Pierre's Overpass query (http://overpass-turbo.eu/s/q5K), and
> zoom/pan to the area of interest.
> * Press Run and let Overpass do its work. Don't be afraid when Overpass
> mentions that the result is huge if you have a modern computer. Last night
> I wasn't experiencing any problems with 30MB data.
> * Press Export, and choose JOSM. Press "Repair query" when Overpass asks
> you to decide.
> * JOSM starts downloading the data. Note that you're only getting the
> outer rings.
> * Go to these rings one by one, and load data (at least you'll need the
> relationship itself).
> * Remove the natural=wood tag from the outer ring.
> * Eventually JOSM starts looking cluttered, because of all the extra data.
> You can use the search query "type:way natural=wood role:outer" to see if
> there are still rings needing work.
> * Upload your changes. Be prepared to handle conflicts if someone else is
> working near you on this issue as well.
> * After a while, check Overpass Turbo again.
>
> I'm not sure what the update frequency is, but Pierre's changes from 4
> hours ago were already processed.
>
> Good luck!
>
> Frank
>
>
> On 01-07-2017 09:52, Jochen Topf wrote:
>
>> On Fri, Jun 30, 2017 at 11:47:36PM +0200, Frank Steggink wrote:
>>
>>> On 30-06-2017 21:21, Jochen Topf wrote:
>>>
 On Fri, Jun 30, 2017 at 08:16:40PM +0200, Frank Steggink wrote:

> Maybe I'm not understanding it, but in the OSM inspector [1] I just
> see one
> case of old style multipolygon, in Manitoba. Last week, when you
> posted your
> original message, I just saw one case in New Brunswick. IIRC, it was a
> park,
> not even from the Canvec import.
>
 The types of problems I am talking about don't show up in the OSM
 inspector. This is not old-style multipolygons (where tags are on the
 outer ways and not on the relation), but multipolygons where the tags
 are on the relation AND on the ways.

>>> Ah, ok, now I understand. Since there was a lot of discussion about old
>>> style multipolygon tagging, and since this type of problem hasn't been
>>> added
>>> to OSM inspector,  this wasn't immediately obvious.
>>>
 In the OSM inspector other errors can be seen, but the most prevalent
> one is
> "Touching rings". Maybe indeed a case of suboptimal mapping, but
> nothing
> which seems urgent to me.
>
> Here is an example of a forest multipolygon, imported by me
> (canvec_fsteggink). It is still version 1, but it has tags on the
> relation,
> not on the rings (except for the quarries): [2]
> This is from Canvec v7.0. IIRC, we started at v6.0, and the last
> version I
> know of is v10.0. Maybe v6.0 had wrong tagging, but I'm not seeing any
> such
> cases in the OSM inspector.
>
> So, I'd like to ask you to give a couple of examples where data
> imported
> from Canvec is clearly wrong with regard to old style multipolygon
> tagging.
>
 Here are all cases in Canada (not only those from the imports):
 https://tmp.jochentopf.com/954226a3acab882d28d8500ddef8203d/
 same-tags-ca.pbf

 Here is one example where you can clearly see the problem:
 http://www.openstreetmap.org/relation/541821

>>> How difficult would it be to add this to OSM inspector? Not everybody has
>>> Postgres running, and is able to use osm2pgsql. Yes, there is
>>> documentation,
>>> but it requires some technical skills. Also, it would be very convenient
>>> to
>>> have this updated daily.
>>>
>> It is not that difficult to add to the OSM Inspector and if I have the
>> time I'll work on that together with the Geofabrik people.
>>
>> When we have clear examples, then it might be easier to come up with a
> plan
> how to fix it. But so far, I see absolutely no reason why Canada
> stands out
> in a negative way. Yes, we all acknowledge that Canvec data is
> suboptimal,
> but as others already have pointed out, mapping everything by hand in
> especially remote areas is nearly impossible.
>
 Canada stands out in a 

Re: [Talk-ca] Multipolygon problems

[Frank Steggink]

Hi Jochen,

Maybe a MapRoulette challenge might even not be necessary. Yesterday I 
started to clean up a bit in Québec, but since it was already past 
midnight for me, I wanted to continue this morning. To my surprise 
Pierre has done a lot of work and now the entire province of Québec 
looks to be free from these errors. I just could find three errors, and 
fixed them. Bon travail, Pierre!


The OSM inspector will still be a good idea, in order to spot future errors.

To all, this is the procedure I used yesterday, and probably something 
similar also by Pierre.

* Not sure if it is a requirement, but it's better to use 64 bit Java.
* You'll need JOSM with the remote control plugin. You'll also need to 
start JOSM.
* Use Pierre's Overpass query (http://overpass-turbo.eu/s/q5K), and 
zoom/pan to the area of interest.
* Press Run and let Overpass do its work. Don't be afraid when Overpass 
mentions that the result is huge if you have a modern computer. Last 
night I wasn't experiencing any problems with 30MB data.
* Press Export, and choose JOSM. Press "Repair query" when Overpass asks 
you to decide.
* JOSM starts downloading the data. Note that you're only getting the 
outer rings.
* Go to these rings one by one, and load data (at least you'll need the 
relationship itself).

* Remove the natural=wood tag from the outer ring.
* Eventually JOSM starts looking cluttered, because of all the extra 
data. You can use the search query "type:way natural=wood role:outer" to 
see if there are still rings needing work.
* Upload your changes. Be prepared to handle conflicts if someone else 
is working near you on this issue as well.

* After a while, check Overpass Turbo again.

I'm not sure what the update frequency is, but Pierre's changes from 4 
hours ago were already processed.


Good luck!

Frank


On 01-07-2017 09:52, Jochen Topf wrote:

On Fri, Jun 30, 2017 at 11:47:36PM +0200, Frank Steggink wrote:

On 30-06-2017 21:21, Jochen Topf wrote:

On Fri, Jun 30, 2017 at 08:16:40PM +0200, Frank Steggink wrote:

Maybe I'm not understanding it, but in the OSM inspector [1] I just see one
case of old style multipolygon, in Manitoba. Last week, when you posted your
original message, I just saw one case in New Brunswick. IIRC, it was a park,
not even from the Canvec import.

The types of problems I am talking about don't show up in the OSM
inspector. This is not old-style multipolygons (where tags are on the
outer ways and not on the relation), but multipolygons where the tags
are on the relation AND on the ways.

Ah, ok, now I understand. Since there was a lot of discussion about old
style multipolygon tagging, and since this type of problem hasn't been added
to OSM inspector,  this wasn't immediately obvious.

In the OSM inspector other errors can be seen, but the most prevalent one is
"Touching rings". Maybe indeed a case of suboptimal mapping, but nothing
which seems urgent to me.

Here is an example of a forest multipolygon, imported by me
(canvec_fsteggink). It is still version 1, but it has tags on the relation,
not on the rings (except for the quarries): [2]
This is from Canvec v7.0. IIRC, we started at v6.0, and the last version I
know of is v10.0. Maybe v6.0 had wrong tagging, but I'm not seeing any such
cases in the OSM inspector.

So, I'd like to ask you to give a couple of examples where data imported
from Canvec is clearly wrong with regard to old style multipolygon tagging.

Here are all cases in Canada (not only those from the imports):
https://tmp.jochentopf.com/954226a3acab882d28d8500ddef8203d/same-tags-ca.pbf

Here is one example where you can clearly see the problem:
http://www.openstreetmap.org/relation/541821

How difficult would it be to add this to OSM inspector? Not everybody has
Postgres running, and is able to use osm2pgsql. Yes, there is documentation,
but it requires some technical skills. Also, it would be very convenient to
have this updated daily.

It is not that difficult to add to the OSM Inspector and if I have the
time I'll work on that together with the Geofabrik people.


When we have clear examples, then it might be easier to come up with a plan
how to fix it. But so far, I see absolutely no reason why Canada stands out
in a negative way. Yes, we all acknowledge that Canvec data is suboptimal,
but as others already have pointed out, mapping everything by hand in
especially remote areas is nearly impossible.

Canada stands out in a negative way, because
a) there are so many problems. Nearly a third of the cases worldwide are in
 Canada and
b) most of these problems are probably caused by one little program, the
 program used to convert/import the CanVec data.

As you might have noticed, later imports, like the example I provided, don't
have that issue anymore. I'm mentioning this to express that not _all_
Canvec data is at fault! Only the first couple of versions. However, for
some reason this was never noticed up until a point that collaborative
action was done to have 

Re: [Talk-ca] Multipolygon problems

[Jochen Topf]

On Fri, Jun 30, 2017 at 11:47:36PM +0200, Frank Steggink wrote:
> On 30-06-2017 21:21, Jochen Topf wrote:
> > On Fri, Jun 30, 2017 at 08:16:40PM +0200, Frank Steggink wrote:
> > > Maybe I'm not understanding it, but in the OSM inspector [1] I just see 
> > > one
> > > case of old style multipolygon, in Manitoba. Last week, when you posted 
> > > your
> > > original message, I just saw one case in New Brunswick. IIRC, it was a 
> > > park,
> > > not even from the Canvec import.
> > The types of problems I am talking about don't show up in the OSM
> > inspector. This is not old-style multipolygons (where tags are on the
> > outer ways and not on the relation), but multipolygons where the tags
> > are on the relation AND on the ways.
> Ah, ok, now I understand. Since there was a lot of discussion about old
> style multipolygon tagging, and since this type of problem hasn't been added
> to OSM inspector,  this wasn't immediately obvious.
> > > In the OSM inspector other errors can be seen, but the most prevalent one 
> > > is
> > > "Touching rings". Maybe indeed a case of suboptimal mapping, but nothing
> > > which seems urgent to me.
> > > 
> > > Here is an example of a forest multipolygon, imported by me
> > > (canvec_fsteggink). It is still version 1, but it has tags on the 
> > > relation,
> > > not on the rings (except for the quarries): [2]
> > > This is from Canvec v7.0. IIRC, we started at v6.0, and the last version I
> > > know of is v10.0. Maybe v6.0 had wrong tagging, but I'm not seeing any 
> > > such
> > > cases in the OSM inspector.
> > > 
> > > So, I'd like to ask you to give a couple of examples where data imported
> > > from Canvec is clearly wrong with regard to old style multipolygon 
> > > tagging.
> > Here are all cases in Canada (not only those from the imports):
> > https://tmp.jochentopf.com/954226a3acab882d28d8500ddef8203d/same-tags-ca.pbf
> > 
> > Here is one example where you can clearly see the problem:
> > http://www.openstreetmap.org/relation/541821
> How difficult would it be to add this to OSM inspector? Not everybody has
> Postgres running, and is able to use osm2pgsql. Yes, there is documentation,
> but it requires some technical skills. Also, it would be very convenient to
> have this updated daily.

It is not that difficult to add to the OSM Inspector and if I have the
time I'll work on that together with the Geofabrik people.

> > > When we have clear examples, then it might be easier to come up with a 
> > > plan
> > > how to fix it. But so far, I see absolutely no reason why Canada stands 
> > > out
> > > in a negative way. Yes, we all acknowledge that Canvec data is suboptimal,
> > > but as others already have pointed out, mapping everything by hand in
> > > especially remote areas is nearly impossible.
> > Canada stands out in a negative way, because
> > a) there are so many problems. Nearly a third of the cases worldwide are in
> > Canada and
> > b) most of these problems are probably caused by one little program, the
> > program used to convert/import the CanVec data.
> As you might have noticed, later imports, like the example I provided, don't
> have that issue anymore. I'm mentioning this to express that not _all_
> Canvec data is at fault! Only the first couple of versions. However, for
> some reason this was never noticed up until a point that collaborative
> action was done to have it fixed. Probably because the rendering pipeline of
> the slippy map was accepting this kind of tagging up until recently.

Okay, that is a big relief already. At least we are not making this
problem worse by new imports that might happen in the future.

> > Mapping Canada "by hand" might be difficult because it is such a huge
> > country and there aren't that many mappers. But the same arguments goes
> > for why you have to be extra careful importing data. If you break
> > something, there are not enough people to fix it manually. And, yes,
> > errors do happen. And if we find them, we fix them and move on. But
> > errors from imports can be so huge there aren't enough people there to
> > fix them manually.
> The world is so huge that there aren't enough people to create and maintain
> a global world map. However, OSM exists. Fixing errors can also be
> crowdsourced. Martijn van Exel is really doing a great job with MapRoulette,
> for instance. Although fixing errors (cleaning up the mess left behind by
> others) is not nearly as rewarding as mapping, it might be easier to do,
> especially since there is no need for a lot of creativity when fixing the
> same kind of errors.

You might have seen that I spent a lot of time in the last months to
create more than 60 Maproulette challenges for all sorts of different
multipolygon problems in different communities. And the community worked
tirelessly on all these problems. (http://area.jochentopf.com/fixed.html)

If the Canadian community steps up and is willing to do this work
manually, I'd he happy to provide such Maproulette