Re: [Talk-ca] Canvec Reverts
I agree that forest that are way too costly in time should be removed, but they should also be replaced with something(better) not just mass removed, oh well we'll get to it later kind of thing On Sep 1, 2016 8:05 PM, "Sam Dyck"wrote: > I took a break to make supper, so I'm going to have to respond > collectively. > > - Sammuell and sammuell_imports are my accounts. Both the forest and the > lake are from Canvec data imported together as in the same tile. I Imported > the whole thing (I would never import over existing data that carelessly), > and take responsibility for that. > > - As my subsequent email showed, I missed the part about water/forest > overlaps in Paul's email. Both and other people on this list have explained > our feelings about this, but I will accept DWGs decision. > > - Per Michael's suggestion: This is constructive, but I do not feel that > it is feasible because of the tiled structure of Canvec. To try and shift > the forest layer over would make the problem worse. > > - Removal of the forest layer may be the best solution here. However there > are many places in where the data matches up perfectly. Speaking only of > areas where I am the primary importer, most of the forests in Ontario west > of Thunder Bay would have to go, which is unfortunate because some have > been manually corrected, though not enough to save them. Much (but > certainly not all) of the forests in Manitoba could be saved with minimal > effort. > > - If/when this is done we could look at ways to restore these forests. In > areas that are mostly forest it shouldn't be too difficult to fill them > using large multipolygons. > > Sam > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Canvec Reverts
I took a break to make supper, so I'm going to have to respond collectively. - Sammuell and sammuell_imports are my accounts. Both the forest and the lake are from Canvec data imported together as in the same tile. I Imported the whole thing (I would never import over existing data that carelessly), and take responsibility for that. - As my subsequent email showed, I missed the part about water/forest overlaps in Paul's email. Both and other people on this list have explained our feelings about this, but I will accept DWGs decision. - Per Michael's suggestion: This is constructive, but I do not feel that it is feasible because of the tiled structure of Canvec. To try and shift the forest layer over would make the problem worse. - Removal of the forest layer may be the best solution here. However there are many places in where the data matches up perfectly. Speaking only of areas where I am the primary importer, most of the forests in Ontario west of Thunder Bay would have to go, which is unfortunate because some have been manually corrected, though not enough to save them. Much (but certainly not all) of the forests in Manitoba could be saved with minimal effort. - If/when this is done we could look at ways to restore these forests. In areas that are mostly forest it shouldn't be too difficult to fill them using large multipolygons. Sam ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
I agree with Frederik here. New imports need to make sure they follow the import guidelines, including using a dedicated import account. The wiki information could all be cleaned up, consolidated, and then new users would know the current status and process for importing new information. For cleaning up the existing information, I like the idea of getting some MapRoulette tasks going, or maybe writing some new validations in osmlint, osmose, etc. Certainly in the BC and Ontario data I've seen, it's not just the forest that is problematic, but also the water layer. Creeks occasionally flow the wrong way (a bad very old import I suspect), lakes and wetlands are mixed/overlaid. Alan (alarobric) On Thu, Sep 1, 2016 at 4:30 PM, john whelanwrote: > >I think a super good first step would be try and ensure that future > imports are done diligently and don't introduce new issues. > > I think that is a reasonable way forward, and I concur with the rest of > your post. > > I think we need to identify which parts of CANVEC are giving concern, each > province is a different mixture of data sources, I suspect it is the forest > and land use that are the most problematic. > > The older tiles had a problem in that a highway would reach the edge of > the tile and there would be a matching highway on the next tile but there > was no connection. I spent many a happy hour, too many of them, merging > nodes so routing would work. I think the cities have been cleaned up but > more rural areas still have the odd one or two to do. > > Probably what would make a lot of sense as a next step is to grab the > provincial OSM dumps, chop them up into manageable portions then load them > up into JOSM and run a modern validation on them. > > Cheerio John > > On 1 September 2016 at 19:13, Frederik Ramm wrote: > >> Andrew, >> >> On 09/02/2016 12:47 AM, Andrew Lester wrote: >> > If people from outside of Canada have decided that our data is so bad >> > that it needs to be completely wiped out in its entirety, then I guess >> > we're going to have to do something drastic to try to prevent this. >> >> I think a super good first step would be try and ensure that future >> imports are done diligently and don't introduce new issues. (This might >> be the "better documentation" step that Paul mentioned.) It really >> shouldn't be too hard to detect whether your planned import causes >> overlapping lakes and forests, but there needs to be an agreement that >> these things matter and that you cannot simply upload "because if CanVec >> says that forest and water overlap then this must be true". >> >> Then one could take stock of existing issues and make a plan on how to >> fix them. >> >> Whether fixing existing issues will necessitate the wholesale removal of >> some imports is something that should be decided down the line; I know >> too little about CanVec imports to say whether some problems are >> systemic in the data source, or certain regions, or just introduced by >> clumsy importers. Any large-scale removal of imported data (perhaps to >> replace it with new, better-imported data) would also have to take into >> account potential manual work that has been performed on the imported by >> mappers with local knowledge and it would be sad to lose that. >> >> Bye >> Frederik >> >> -- >> Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" >> >> ___ >> Talk-ca mailing list >> Talk-ca@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk-ca >> > > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
>I think a super good first step would be try and ensure that future imports are done diligently and don't introduce new issues. I think that is a reasonable way forward, and I concur with the rest of your post. I think we need to identify which parts of CANVEC are giving concern, each province is a different mixture of data sources, I suspect it is the forest and land use that are the most problematic. The older tiles had a problem in that a highway would reach the edge of the tile and there would be a matching highway on the next tile but there was no connection. I spent many a happy hour, too many of them, merging nodes so routing would work. I think the cities have been cleaned up but more rural areas still have the odd one or two to do. Probably what would make a lot of sense as a next step is to grab the provincial OSM dumps, chop them up into manageable portions then load them up into JOSM and run a modern validation on them. Cheerio John On 1 September 2016 at 19:13, Frederik Rammwrote: > Andrew, > > On 09/02/2016 12:47 AM, Andrew Lester wrote: > > If people from outside of Canada have decided that our data is so bad > > that it needs to be completely wiped out in its entirety, then I guess > > we're going to have to do something drastic to try to prevent this. > > I think a super good first step would be try and ensure that future > imports are done diligently and don't introduce new issues. (This might > be the "better documentation" step that Paul mentioned.) It really > shouldn't be too hard to detect whether your planned import causes > overlapping lakes and forests, but there needs to be an agreement that > these things matter and that you cannot simply upload "because if CanVec > says that forest and water overlap then this must be true". > > Then one could take stock of existing issues and make a plan on how to > fix them. > > Whether fixing existing issues will necessitate the wholesale removal of > some imports is something that should be decided down the line; I know > too little about CanVec imports to say whether some problems are > systemic in the data source, or certain regions, or just introduced by > clumsy importers. Any large-scale removal of imported data (perhaps to > replace it with new, better-imported data) would also have to take into > account potential manual work that has been performed on the imported by > mappers with local knowledge and it would be sad to lose that. > > Bye > Frederik > > -- > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Andrew, On 09/02/2016 12:47 AM, Andrew Lester wrote: > If people from outside of Canada have decided that our data is so bad > that it needs to be completely wiped out in its entirety, then I guess > we're going to have to do something drastic to try to prevent this. I think a super good first step would be try and ensure that future imports are done diligently and don't introduce new issues. (This might be the "better documentation" step that Paul mentioned.) It really shouldn't be too hard to detect whether your planned import causes overlapping lakes and forests, but there needs to be an agreement that these things matter and that you cannot simply upload "because if CanVec says that forest and water overlap then this must be true". Then one could take stock of existing issues and make a plan on how to fix them. Whether fixing existing issues will necessitate the wholesale removal of some imports is something that should be decided down the line; I know too little about CanVec imports to say whether some problems are systemic in the data source, or certain regions, or just introduced by clumsy importers. Any large-scale removal of imported data (perhaps to replace it with new, better-imported data) would also have to take into account potential manual work that has been performed on the imported by mappers with local knowledge and it would be sad to lose that. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
If people from outside of Canada have decided that our data is so bad that it needs to be completely wiped out in its entirety, then I guess we're going to have to do something drastic to try to prevent this. @Michael, Frederik, Paul: would it be good enough for us to wipe out any and all CanVec forest data (or even ALL forest data because some could have been derived from CanVec)? This seems to be the biggest cause for concern. If not, what do you think we need to do to prevent all CanVec data from being wiped out? Wiping out all CanVec data would effectively clear out the majority of the Canadian map and really isn't an option in our minds. Do we need to get rid of all forest data and then go on a cooperative fixing blitz (maybe using MapRoulette or Tasking Manager) to fix every single JOSM validator error across the country? In short, if we're doing things so wrong, what can we do to make things right other than have a German revert all of our changesets so we can start from scratch? Andrew Victoria, BC, Canada - Original Message - From: "Sam Dyck" <samueld...@gmail.com> To: "Talk-CA OpenStreetMap" <talk-ca@openstreetmap.org> Sent: Thursday, September 1, 2016 2:38:38 PM Subject: Re: [Talk-ca] CanVec Reverts After reading Paul's email again, its possible that what Nakaner is doing is in line with Paul's suggestion, if unnecessarily confrontational. I tried to play around in JOSM to see if I could get the forest polygons to a point where Nakaner would leave us alone by mercilessly deleting all of the inner ways in the forest multipolygons, but because of the way things are structured around rivers that would be several hours worth of work for one tile. Given this perhaps the only solution is to bulk delete all Canvec forest data. As someone who actually finds the forest data useful this would be extremely unfortunate, but if it allows us to continue imports without excessive external scrutiny then I am willing to except it. (apologies for the English only emails, my French writing skills are sadly lacking) On Thu, Sep 1, 2016 at 4:20 PM, Pierre Béland < pierz...@yahoo.fr > wrote: L'idéal préparer Une réponse standard indiquant que l'Import Canvec par la communauté OSM Canada est itératif et nous nous assurons collectivement d'améliorer les données. Voir page Wiki Import Canvec et venir discuter sur Talk-Ca si vous avez d'autres questions. Pierre De : Sam Dyck < samueld...@gmail.com > À : Talk-CA OpenStreetMap < talk-ca@openstreetmap.org > Envoyé le : jeudi 1 Septembre 2016 17h06 Objet : Re: [Talk-ca] CanVec Reverts I received the following changeset comment from Nakaner for a Canvec import (changeset 38158126) at 15:55 Central Time (20:55 UTC): "This changeset has uploaded data which does not fit to each other. There is an offset between the water areas and the forest areas. Example: https://www.openstreetmap.org/ way/406539219 Could you please fix this?" I believe the given what we have just spent the last 24 hours discussing this request is unreasonable and the issue is not significant. Thoughts? Sam ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Sam, On 09/01/2016 11:26 PM, Frederik Ramm wrote: >> I believe the given what we have just spent the last 24 hours discussing >> this request is unreasonable and the issue is not significant. Thoughts? > > An importer who imports data into OSM that doesn't match up with already > existing data and doesn't notice it is careless and should do a better job. > > An importer who is asked to fix non-matching data he has accidentally > imported and responds that the request is "unreasonable" should have his > account taken away. It appears I have mistakenly thought that you, Sam, and the import user "sammuell" were one and the same. Apologies. I still think that if the importer cannot be bothered to fix it, the data should be removed until someone has the time to do it right (rather than keeping bad data in OSM until someone can fix it). Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
After reading Paul's email again, its possible that what Nakaner is doing is in line with Paul's suggestion, if unnecessarily confrontational. I tried to play around in JOSM to see if I could get the forest polygons to a point where Nakaner would leave us alone by mercilessly deleting all of the inner ways in the forest multipolygons, but because of the way things are structured around rivers that would be several hours worth of work for one tile. Given this perhaps the only solution is to bulk delete all Canvec forest data. As someone who actually finds the forest data useful this would be extremely unfortunate, but if it allows us to continue imports without excessive external scrutiny then I am willing to except it. (apologies for the English only emails, my French writing skills are sadly lacking) On Thu, Sep 1, 2016 at 4:20 PM, Pierre Béland <pierz...@yahoo.fr> wrote: > L'idéal préparer Une réponse standard indiquant que l'Import Canvec par la > communauté OSM Canada est itératif et nous nous assurons collectivement > d'améliorer les données. Voir page Wiki Import Canvec et venir discuter sur > Talk-Ca si vous avez d'autres questions. > > > Pierre > > > -- > *De :* Sam Dyck <samueld...@gmail.com> > *À :* Talk-CA OpenStreetMap <talk-ca@openstreetmap.org> > *Envoyé le :* jeudi 1 Septembre 2016 17h06 > *Objet :* Re: [Talk-ca] CanVec Reverts > > I received the following changeset comment from Nakaner for a Canvec > import (changeset > 38158126) at 15:55 Central Time (20:55 UTC): > "This changeset has uploaded data which does not fit to each other. There > is an offset between the water areas and the forest areas. Example: > https://www.openstreetmap.org/ > way/406539219 <https://www.openstreetmap.org/way/406539219> > Could you please fix this?" > I believe the given what we have just spent the last 24 hours discussing > this request is unreasonable and the issue is not significant. Thoughts? > Sam > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > > > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Hi Sam, Am 01.09.2016 um 23:06 schrieb Sam Dyck: > I received the following changeset comment from Nakaner for a Canvec import > (changeset > 38158126) at 15:55 Central Time (20:55 UTC): > > "This changeset has uploaded data which does not fit to each other. There > is an offset between the water areas and the forest areas. Example: > https://www.openstreetmap.org/way/406539219 > > Could you please fix this?" > > I believe the given what we have just spent the last 24 hours discussing > this request is unreasonable and the issue is not significant. Thoughts? If you have a proper look at the whole area around, you will see that there is a systematic offset between the water "layer" and the forest layer. The forest layer should be moved about 10 to 30 metres to northwest. I wonder how such an error can be overseen during the preparation of this tile. Other examples: https://www.openstreetmap.org/way/402043390 https://www.openstreetmap.org/#map=16/49.3997/-87.4329 Maybe the original data was traced from different aerial imagery and that's why there is an offset which is not constant. Best regards Michael -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) signature.asc Description: OpenPGP digital signature ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Hi, On 2016-09-01 21:06:57, Sam Dyck wrote: > I believe the given what we have just spent the last 24 hours discussing > this request is unreasonable and the issue is not significant. Thoughts? An importer who imports data into OSM that doesn't match up with already existing data and doesn't notice it is careless and should do a better job. An importer who is asked to fix non-matching data he has accidentally imported and responds that the request is "unreasonable" should have his account taken away. (This is my personal opinion. DWG is more nuanced but Paul Norman's message which you have read says clearly that the importer has the responsibility to see that the data matches up with existing stuff.) Either https://www.openstreetmap.org/way/406539219 is wrong, or the forest around it is wrong. And to quote someone else from this thread, the result is certainly not aesthetically pleasing either. You should find out which, and take appropriate steps. I am speechless to hear that not only are you importing broken data, but you also consider it unreasonable to fix it. If you can't be bothered to care for quality when importing, then don't, and wait for someone more responsible to come along. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
I received the following changeset comment from Nakaner for a Canvec import (changeset 38158126) at 15:55 Central Time (20:55 UTC): "This changeset has uploaded data which does not fit to each other. There is an offset between the water areas and the forest areas. Example: https://www.openstreetmap.org/way/406539219 Could you please fix this?" I believe the given what we have just spent the last 24 hours discussing this request is unreasonable and the issue is not significant. Thoughts? Sam ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
>From what Rps333 told me in person he had worked many hours on that changeset and was frustrated when someone reverted before fixes could be applied. Could you revert the revert? On Sep 1, 2016 4:09 PM, "Paul Norman"wrote: > Hello, > > Multiple people have referred this issue and changeset 39517002 to the > OpenStreetMap Foundation Data Working Group about this issue. The Data > Working Group has responsibility for the resolution of disputes beyond > the normal means of the community, as well as some responsibilities > concerning imports. I am replying here rather than the changeset as it > should reach everyone involved. > > CanVec Quality > == > > The CanVec import is one that was started long ago so parts of the > documentation are lacking and some aspects are unclear.[1] CanVec, like > any other import, has certain hard requirements, including the use of a > dedicated user account. > > In particular it is expected that imported CanVec data is > > - Integrated with existing data, particularly at NTS/CanVec tile edges > > - Valid > > - Internally consistent with both the newly imported CanVec and existing > OSM data, particularly to avoid overlapping landuse like forest and > water, forest and residential, or wetlands in what is obviously a > residential subdivision. > > - Uploaded in small enough parts that the changesets make sense. This > means never uploading more than 50k objects at once, and typically > fewer than 10k. > > - Not duplicating existing OSM data. Evaluating what data is better > generally requires experience with both CanVec and OSM data in the > *region being mapped* > > It is not required that CanVec data is compared an external source like > Bing imagery, but this is helpful, particularly when resolving problems. > > We encourage the Canadian community to develop better documentation for > people importing CanVec to achieve this and to remove outdated > documentation.[2] > > Reverting > = > > Advance permission is not required for reverts, nor for normal mapping > activities. At the same time, users are expected to be responsible, > particularly when using tools for reverting which allow large-scale > changes where other users may disagree with them. > > Where there are problems with an import reverting is an option, but > just one of many, and often not the appropriate first action. Unless > there are legal problems[3] or fatal problems with the import it is > preferable if the original importer can fix the problems in a timely > manner. There was every indication this was going to happen in this case. > > The revert of 39517002 was inappropriate and counter-productive. New > actions like this revert may lead to further Data Working Group > involvement and potentially blocks. If the Canadian community needs help > reverting 41749133 and 41756737, the Data Working Group can revert those > changesets. > > While not going into depth on the changeset discussion at this time, I > want to remind everyone involved that OpenStreetMap is a crowd-sourcing > project, which inherently involves working with other people. This > requires good communication, which was absent here. > > Paul Norman > > For the OpenStreetMap Foundation Data Working Group > > [1]: Except for the CanVec license, which is ODbL and possibly CC BY-SA > > [2]: Much of the CanVec documentation is outdated, which makes it > difficult to know what is relevant. A good start would be removing > outdated documentation. > > [3]: Please refer cases of large-scale infringement to > d...@osmfoundation.org > so we can "redact" the content to remove it from publicly viewable > history. > > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Canvec data has been imported for many years so why the sudden challenge and change? Are you willing to reimport the CANVEC data or are you intent on leaving a blank area in the map? Cheerio John On 1 September 2016 at 15:41, Michael Reichertwrote: > Hi James, > > Am 2016-09-01 um 15:04 schrieb James: > > To blatantly toss discussions of the > > past whether to import CanVec or not into OSM and *that was approved*, … > > Could you point me to the discussion at the Imports mailing list? (link > to the archive of the mailing list) > > I am not against importing CanVec data, I am just against importing > CanVec data in a bad way. > > Best regards > > Michael > > -- > Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten > ausgenommen) > I prefer GPG encryption of emails. (does not apply on mailing lists) > > > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Hello, Multiple people have referred this issue and changeset 39517002 to the OpenStreetMap Foundation Data Working Group about this issue. The Data Working Group has responsibility for the resolution of disputes beyond the normal means of the community, as well as some responsibilities concerning imports. I am replying here rather than the changeset as it should reach everyone involved. CanVec Quality == The CanVec import is one that was started long ago so parts of the documentation are lacking and some aspects are unclear.[1] CanVec, like any other import, has certain hard requirements, including the use of a dedicated user account. In particular it is expected that imported CanVec data is - Integrated with existing data, particularly at NTS/CanVec tile edges - Valid - Internally consistent with both the newly imported CanVec and existing OSM data, particularly to avoid overlapping landuse like forest and water, forest and residential, or wetlands in what is obviously a residential subdivision. - Uploaded in small enough parts that the changesets make sense. This means never uploading more than 50k objects at once, and typically fewer than 10k. - Not duplicating existing OSM data. Evaluating what data is better generally requires experience with both CanVec and OSM data in the *region being mapped* It is not required that CanVec data is compared an external source like Bing imagery, but this is helpful, particularly when resolving problems. We encourage the Canadian community to develop better documentation for people importing CanVec to achieve this and to remove outdated documentation.[2] Reverting = Advance permission is not required for reverts, nor for normal mapping activities. At the same time, users are expected to be responsible, particularly when using tools for reverting which allow large-scale changes where other users may disagree with them. Where there are problems with an import reverting is an option, but just one of many, and often not the appropriate first action. Unless there are legal problems[3] or fatal problems with the import it is preferable if the original importer can fix the problems in a timely manner. There was every indication this was going to happen in this case. The revert of 39517002 was inappropriate and counter-productive. New actions like this revert may lead to further Data Working Group involvement and potentially blocks. If the Canadian community needs help reverting 41749133 and 41756737, the Data Working Group can revert those changesets. While not going into depth on the changeset discussion at this time, I want to remind everyone involved that OpenStreetMap is a crowd-sourcing project, which inherently involves working with other people. This requires good communication, which was absent here. Paul Norman For the OpenStreetMap Foundation Data Working Group [1]: Except for the CanVec license, which is ODbL and possibly CC BY-SA [2]: Much of the CanVec documentation is outdated, which makes it difficult to know what is relevant. A good start would be removing outdated documentation. [3]: Please refer cases of large-scale infringement to d...@osmfoundation.org so we can "redact" the content to remove it from publicly viewable history. ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Hum First discussions about importing NRCan data can be find here... https://lists.openstreetmap.org/pipermail/talk-ca/2008-November/000228.html They are talking about importing GeoBase - which is exactly the same content that is found in Canvec since Canvec is the merging of GeoBase layers. If you look at the date (November 2008), the import mailing list even did not exist. We can go that way but I do not think it the point here. We must talk about deleting data that was imported in good faith, without leaving the community enough time to correct them. Daniel -Original Message- From: Michael Reichert [mailto:naka...@gmx.net] Sent: Thursday, 1 September, 2016 15:41 To: James Cc: talk-ca@openstreetmap.org Subject: Re: [Talk-ca] CanVec Reverts Hi James, Am 2016-09-01 um 15:04 schrieb James: > To blatantly toss discussions of the > past whether to import CanVec or not into OSM and *that was approved*, > … Could you point me to the discussion at the Imports mailing list? (link to the archive of the mailing list) I am not against importing CanVec data, I am just against importing CanVec data in a bad way. Best regards Michael -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
I'm usually just a lurker on these lists but this has dragged me out of my cave. Michael, I'm glad you care about the quality of the map, I do too. I welcome you to take an constructive approach to working with these problems. Canada has an area of 9,984,670 square kilometers with a population density of only 8.8. That presents us with challenges that wouldn't apply to a country with an area of 357,022 square kilometers and a population density of 593. Rather than handing out an ultimatum to the Canadian mapping community how about you work with us? We share your concerns in regards to data quality but your unilateral reversion of commits without communication or cooperation is damaging to the map. I find your comment "if it has not been touched for a few weeks" comment to be insulting and it shows a complete lack of understanding of the realities on the ground. I'm personally working at improving the map in my area (County of Vermilion River, Alberta, Canada) and you are more than welcome to constructively help. However, just because I haven't touched something for a couple of weeks doesn't mean I've forgotten about it. It means I was on holidays or got busy with other things in the office. Have you ever tried importing data using JOSM? I've spent hours poring over a few square miles of countryside and trying to get everything cleaned up and merged. Then, when I actually import, the data gets split into much smaller sizes. Or maybe I get some of it imported and then have to fix a bug I missed in something else. To somebody like you it looks like I just dumped a bunch of data in when in reality I've been picking away at it for a few days. Darren Wiebe On Thu, Sep 1, 2016 at 8:49 AM, Sam Dyck <samueld...@gmail.com> wrote: > Micheal > > Thanks for contacting us. I must object strongly to your use of the Worst > of OSM example and generally assumption that the data is broken if it > doesn't line up. I checked multiple commercial imagery providers before I > found a digitalglobe image that covered the area during the summer. There > is a large patch of sand between the vegetation-filled area and the coast. > As for the boundary, that comes from another official source, I think it is > supposed to be spaced off of the coastline, though I don't remember exactly > how they calculated it, we would likely need a constitutional change to > make it line up with the coast. Just because things don't match up does not > mean that the data is wrong. Nature doesn't always translate into nice, > clean maps. > > Sam > > -Original Message- > From: Michael Reichert [mailto:naka...@gmx.net] > Sent: Thursday, 1 September, 2016 01:39 > To: talk-ca@openstreetmap.org > Subject: [Talk-ca] CanVec Reverts > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi, > > unfortunately posting via Gmane does not seem to work (the website is down > but NNTP still works), that's why I have to start a new thread. :-( > > Am Tue, 30 Aug 2016 21:41:21 -0500 schrieb Sam Dyck: > > After reading through the changeset discussion, I discovered that one > > of my imports in Northern Manitoba made Worst of OSM. > > (http://worstofosm.tumblr.com/post/22180046353/dear- > > openstreetmap-isnt-it-strange-how-the). As someone who spends a some > > time amount of time in some of relatively unpopulated areas of Canada > > and makes an effort to check the quality of Canvec data (which is > > usually pretty good), I do agree that it is impossible to do > > everything to the same level of quality that we would provide in > > Toronto or Timmins or even small prairie towns. > > First of all, it is ok that an import takes a few years and therefore > creates ugly green rectancles on the map. If an import is "unavoidable" > :-), a manual import is the best thing that can be happen. But if someone > uploads a changeset without a manual review beforehand, he counteracts the > aim of a manual import: addind good data to OpenStreetMap. That's what I am > mainly fighting against. If a users uploads much more than 100 objects per > minute [1], you can be sure that he has not done any manual review. A > manual review by myself confirmed this these. I am fighting against such > changesets/users. > > A good imports must be reviewed *before* it is being uploaded. The review > contains: > - - Run JOSM validator, fix all warnings and errors. This includes all > warnings regarding validity of areas. (you can argue if all warnings about > "deprecated" tagging have to be fixed) > - - Compare the data with available imagery. Is the forest really a forest > or is another tag more appropiate? Right-click on a Bing tile at JOSM and > have a look how old/recent the imagery is. > - - Check if CanVec data fits to its
Re: [Talk-ca] CanVec Reverts
Micheal Thanks for contacting us. I must object strongly to your use of the Worst of OSM example and generally assumption that the data is broken if it doesn't line up. I checked multiple commercial imagery providers before I found a digitalglobe image that covered the area during the summer. There is a large patch of sand between the vegetation-filled area and the coast. As for the boundary, that comes from another official source, I think it is supposed to be spaced off of the coastline, though I don't remember exactly how they calculated it, we would likely need a constitutional change to make it line up with the coast. Just because things don't match up does not mean that the data is wrong. Nature doesn't always translate into nice, clean maps. Sam -Original Message- From: Michael Reichert [mailto:naka...@gmx.net] Sent: Thursday, 1 September, 2016 01:39 To: talk-ca@openstreetmap.org Subject: [Talk-ca] CanVec Reverts -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, unfortunately posting via Gmane does not seem to work (the website is down but NNTP still works), that's why I have to start a new thread. :-( Am Tue, 30 Aug 2016 21:41:21 -0500 schrieb Sam Dyck: > After reading through the changeset discussion, I discovered that one > of my imports in Northern Manitoba made Worst of OSM. > (http://worstofosm.tumblr.com/post/22180046353/dear- > openstreetmap-isnt-it-strange-how-the). As someone who spends a some > time amount of time in some of relatively unpopulated areas of Canada > and makes an effort to check the quality of Canvec data (which is > usually pretty good), I do agree that it is impossible to do > everything to the same level of quality that we would provide in > Toronto or Timmins or even small prairie towns. First of all, it is ok that an import takes a few years and therefore creates ugly green rectancles on the map. If an import is "unavoidable" :-), a manual import is the best thing that can be happen. But if someone uploads a changeset without a manual review beforehand, he counteracts the aim of a manual import: addind good data to OpenStreetMap. That's what I am mainly fighting against. If a users uploads much more than 100 objects per minute [1], you can be sure that he has not done any manual review. A manual review by myself confirmed this these. I am fighting against such changesets/users. A good imports must be reviewed *before* it is being uploaded. The review contains: - - Run JOSM validator, fix all warnings and errors. This includes all warnings regarding validity of areas. (you can argue if all warnings about "deprecated" tagging have to be fixed) - - Compare the data with available imagery. Is the forest really a forest or is another tag more appropiate? Right-click on a Bing tile at JOSM and have a look how old/recent the imagery is. - - Check if CanVec data fits to itself. http://worstofosm.tumblr.com/post/22180046353/dear-openstreetmap-isnt- it-strange-how-the <http://worstofosm.tumblr.com/post/22180046353/dear-openstreetmap-isnt-it-strange-how-the> - - Check if there has been any other data before. If yes, adapt the either the CanVec data or the old data. https://wiki.openstreetmap.org/wiki/File:Import-Fails-Powerlines-Not-Ins ide-Cutting.png <https://wiki.openstreetmap.org/wiki/File:Import-Fails-Powerlines-Not-Inside-Cutting.png> https://www.openstreetmap.org/way/439631732 - - Ways should not overlap with other ways if it is not necessary. The outer ring of a lake should also be inner member of the forest multipolygon. Maybe the program which created the OSM files should be imprved? - - Keep the history. https://wiki.openstreetmap.org/wiki/Good_practice#Keep_the_history If a tile has been imported without being checked manually and no post-upload fixes have been done (i.e. upload without any checks), I will not shrink from reverting it. If a tile has been uploaded to OSM without a review and if it has not been fixed within a month, it is worthless and can easily be reimported at a later time if someone has the time to check and fix it. For the future, I will abstain from reverting changesets which have been imported before September 1, 2016 and whose users are currently doing the fixes that should already have been done. But if I come across an imported tile of low quality which has not been touched for a few weeks and is full of errors, it is just a question of time until it is reverte d. Best regards Michael [1] I had a look on a few of my changesets which added a large number of buildings to OSM. The fastest changeset contained about 60 objects per minute and was full of missing buildings as I later detected while collecting the housenumbers and usage of the buildings. ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Canvec reverts
ne wrong, how to edit and >>> how to fix them. :-/ >>> >>> > 2) A addr:interp way may be split in 2 parts. 2 consequences: >>> > - the interpolation way become useless because it's now 2 different >>> objects. >>> > - the mid-point becomes 2 superposed nodes. Useless duplication. >>> > >>> > 3) A grid tile has a fixed size. It may be appropriate for unpopulated >>> areas >>> > but it is too large for urban areas where editors work at a high zoom >>> level >>> > (17 and up). It's easy to damage a relation without knowing it. >>> > >>> > But there are other problems (not related to the grid): >>> > 4) the relations seem to be designed to be stand-alone. Thus the >>> relation >>> > borders don't share a way. They use 2 superposed ways. Useless >>> duplication. >>> > It's very confusing and we frequently alter the wrong way. >>> > >>> > 5) lakes are represented by 2 superposed identical objects, one for >>> the hole >>> > and one for the lake. If the hole happen to be on top, the name will >>> not be >>> > displayed. It's an unjustifiable complexity for the casual user. >>> > I've also seen triplicated contour (one for the lake, on for the inner >>> role >>> > and one for the outer role) >>> > >>> > Yes, all these quirks can be fixed manually but it's time-consuming >>> and error- >>> > prone. >>> >>> What about reverting the tiles which have all these issues and seem to >>> be uploaded with too few checks beforehand, run a better version of the >>> preprocessor on the CanVec raw data and reimport them again? (That >>> causes a loss of OSM history but an import changeset is not as much >>> valueable than a manual changeset) >>> >>> > Ideally, the contour of a forest must not split any object and it's not >>> > possible with a grid. >>> > The sole advantage of a grid IMHO is to simplify the CanVec exports. >>> >>> What about replacing the grid by less artificial borders, e.g. roads, >>> rivers, powerlines etc. If an area has too few of theses objects, >>> artifical borders could be automatically drawn which are optimized to >>> cut as few objects as possible into two parts. >>> >>> > Some years ago I would have proposed that someone write a guide "How >>> to fix a >>> > CanVec import". But now I would rather propose that someone write a >>> "How to >>> > pre-process a CanVec export before importing it into OSM". >>> >>> +1 >>> >>> All ongoing changesets which import CanVec data should either use an >>> improved version of the preprocessor or should undergo the manual >>> quality checks I described in my other emails and the changeset comments. >>> >>> Best regards >>> >>> Michael >>> >>> >>> -- >>> Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten >>> ausgenommen) >>> I prefer GPG encryption of emails. (does not apply on mailing lists) >>> >>> >>> >>> ___ >>> Talk-ca mailing list >>> Talk-ca@openstreetmap.org >>> https://lists.openstreetmap.org/listinfo/talk-ca >>> >>> >> >> ___ >> Talk-ca mailing list >> Talk-ca@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk-ca >> >> > > -- Forwarded message -- > From: Begin Daniel <jfd...@hotmail.com> > To: Michael Reichert <naka...@gmx.net>, Begin Daniel <jfd...@hotmail.com>, > "talk-ca@openstreetmap.org" <talk-ca@openstreetmap.org> > Cc: > Date: Thu, 1 Sep 2016 13:04:30 + > Subject: Re: [Talk-ca] CanVec Reverts > I understand your point. You might be right about the time between > changesets (even though it may depends on if the users is working with > layers), but I maintain my points about the time it may take within a > changeset. > Daniel > > -Original Message- > From: Michael Reichert [mailto:naka...@gmx.net] > Sent: Thursday, 1 September, 2016 08:37 > To: Begin Daniel; talk-ca@openstreetmap.org > Subject: Re: [Talk-ca] CanVec Reverts > > Hi Daniel, > > Am 2016-09-01 um 12:26 schrieb Begin Daniel: > > Furthermore, I hope you will not use you 100 objects per minute to > >
Re: [Talk-ca] CanVec Reverts
I understand your point. You might be right about the time between changesets (even though it may depends on if the users is working with layers), but I maintain my points about the time it may take within a changeset. Daniel -Original Message- From: Michael Reichert [mailto:naka...@gmx.net] Sent: Thursday, 1 September, 2016 08:37 To: Begin Daniel; talk-ca@openstreetmap.org Subject: Re: [Talk-ca] CanVec Reverts Hi Daniel, Am 2016-09-01 um 12:26 schrieb Begin Daniel: > Furthermore, I hope you will not use you 100 objects per minute to > decide whether or not you will delete a changeset. I think this > threshold is value doesn't' apply (see below) > > Daniel > > About the100 objects threshold. > From my experience, if I load a Canvec tile in JOSM, make all the necessary > corrections and then import the result to OSM, I throw up to 25K objects to > the database within five minutes. As far as I know, the timestamps attached > to the changeset and to the objects is generated by the OSM database when > receiving the data. The five minutes it takes to upload the data to the > database (5K objects per minute) do not reflect the time I spent editing the > data prior to the upload. That's the base of my calculation I did with Rps333's changesets: changeset start end object count 3951757119:30:5319:32:564311 3951768619:35:3019:41:1211724 3951794419:45:1519:47:274963 3951814719:53:2520:04:5519286 As you see, he took less than three minutes minutes after uploading 39517571 to prepare 39517686. You cannot check such an amount of data very well within that time. Best regards Michael -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, Am 2016-09-01 um 13:22 schrieb john whelan: > In many parts of the country there are no local mappers for many > miles or kilometers if you prefer. We do have some very > experienced GIS people around and I'm under the impression that we > really do know what we are doing. This is no carte blanche to import any dataset you can get. The import discussion exists to ensure that an import is done the right way, i.e. importing good data in a good way. If the users who import a dataset base on an import disussion/proposal do not care about the quality, they violate the proposal and therefore the policy because the import becomes an import which has not been approved. Best regards Michael - -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) -BEGIN PGP SIGNATURE- iQIcBAEBCAAGBQJXyCKEAAoJEB87G9rMCMyIqvgQAK58kt5ULsvv0nf0MnCm6ahM Iq8Kub3bLS2ZE93kVlgUl43cEJ6pzr3/XS0fd4hbRPuybektD0N9kNumm/KVZ2Pn 2/fZvOpai3YcVRAiSMZ2HUHaZnz9CYI56xsFYJcE8+RXcmTBXbp1WBmbqd4j7ceu +IRdkrxweAQEDymlYtfI1rd1gA+CJBWnfcRr7Fq1CUNi2yI4M4U697qxtK1TdQuW 4Y+SmiDlUvGJLos0qjzucXs3zatY5SELY3/sTZ4iS0Emla8m5Eq1Wqo09FCGngrT Yov1gQQBuMlUl80BsILM6beAKfhYq0hYgje77VzONmZYN79mMzkXHD3IJS2/lVfZ r4pU2BL6bDTYues0diPefNbCvTi0SkLAOJccsy4+6OLWQ5B9WJgW8yT3KTbv6N0T 25yuaEGBdbLcs4dacZj1PdMKy2W4EgTy2UQKadlc4l4DAVJYyuaLifx0ij8n5pgs hepMWWTh0B5WyIckDGVMBUk9awQFu1IN7gm5zJWgTap6Kz3m0O0TbvOGtaXjod7C teFq+MmZ7wf/ocGc0AMCweZgJUBKiTu+tcKYrFUQq4f6XSCblzYiSJA95NlRNGJh BiLVgu/K7nJ1fYgPhN9wSVGgP21HRs80X2ZVtGLbTL/hZTjSDKNLd8sS6tT+86Ie ek7VLR9LDoAeihcRu3zU =Ea1G -END PGP SIGNATURE- ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
What tells you he didnt prepare batches outside the upload time(offline)? Your logic is skewed On Sep 1, 2016 8:39 AM, "Michael Reichert"wrote: > Hi Daniel, > > Am 2016-09-01 um 12:26 schrieb Begin Daniel: > > Furthermore, I hope you will not use you 100 objects per minute to > decide whether or not you will delete a changeset. I think this threshold > is value doesn't' apply (see below) > > > > Daniel > > > > About the100 objects threshold. > > From my experience, if I load a Canvec tile in JOSM, make all the > necessary corrections and then import the result to OSM, I throw up to 25K > objects to the database within five minutes. As far as I know, the > timestamps attached to the changeset and to the objects is generated by the > OSM database when receiving the data. The five minutes it takes to upload > the data to the database (5K objects per minute) do not reflect the time I > spent editing the data prior to the upload. > > That's the base of my calculation I did with Rps333's changesets: > > changeset start end object count > > 3951757119:30:5319:32:564311 > 3951768619:35:3019:41:1211724 > 3951794419:45:1519:47:274963 > 3951814719:53:2520:04:5519286 > > As you see, he took less than three minutes minutes after uploading > 39517571 to prepare 39517686. You cannot check such an amount of data > very well within that time. > > Best regards > > Michael > > > -- > Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten > ausgenommen) > I prefer GPG encryption of emails. (does not apply on mailing lists) > > > ___ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca > > ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Hi Daniel, Am 2016-09-01 um 12:26 schrieb Begin Daniel: > Furthermore, I hope you will not use you 100 objects per minute to decide > whether or not you will delete a changeset. I think this threshold is value > doesn't' apply (see below) > > Daniel > > About the100 objects threshold. > From my experience, if I load a Canvec tile in JOSM, make all the necessary > corrections and then import the result to OSM, I throw up to 25K objects to > the database within five minutes. As far as I know, the timestamps attached > to the changeset and to the objects is generated by the OSM database when > receiving the data. The five minutes it takes to upload the data to the > database (5K objects per minute) do not reflect the time I spent editing the > data prior to the upload. That's the base of my calculation I did with Rps333's changesets: changeset start end object count 3951757119:30:5319:32:564311 3951768619:35:3019:41:1211724 3951794419:45:1519:47:274963 3951814719:53:2520:04:5519286 As you see, he took less than three minutes minutes after uploading 39517571 to prepare 39517686. You cannot check such an amount of data very well within that time. Best regards Michael -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) signature.asc Description: OpenPGP digital signature ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] CanVec Reverts
Note to Michael Reichert I think you should respect the views of the local community. We know the data isn't perfect but for the most part its good data and as Daniel has said when I've imported the work is done in JOSM off line over a period of time so the time apparently taken doesn't really indicate this. In many parts of the country there are no local mappers for many miles or kilometers if you prefer. We do have some very experienced GIS people around and I'm under the impression that we really do know what we are doing. Cheerio John On 1 September 2016 at 06:26, Begin Daniel <jfd...@hotmail.com> wrote: > Thank for contacting the Canadian community Michael, > You provided us with a short but useful reminder of current rules we > should apply when importing data (or even just making standard edits). > > However, I understand from your last paragraph that you will keep deleting > changesets. I was hoping your email started a discussion on best practices > that we could be put in place in our context since adjusting Canvec data to > the latest rules is a daunting task. I rather feel it is an ultimatum. > > Do your future actions will apply to the imports made a few months, a few > years ago, which are 'full of errors' and for which nobody seems to care? > Are you going to check with concerned contributors (old/future imports) if > they bother or not to see their work deleted before you do it? > > Furthermore, I hope you will not use you 100 objects per minute to decide > whether or not you will delete a changeset. I think this threshold is value > doesn't' apply (see below) > > Daniel > > About the100 objects threshold. > From my experience, if I load a Canvec tile in JOSM, make all the > necessary corrections and then import the result to OSM, I throw up to 25K > objects to the database within five minutes. As far as I know, the > timestamps attached to the changeset and to the objects is generated by the > OSM database when receiving the data. The five minutes it takes to upload > the data to the database (5K objects per minute) do not reflect the time I > spent editing the data prior to the upload. > > -Original Message- > From: Michael Reichert [mailto:naka...@gmx.net] > Sent: Thursday, 1 September, 2016 01:39 > To: talk-ca@openstreetmap.org > Subject: [Talk-ca] CanVec Reverts > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi, > > unfortunately posting via Gmane does not seem to work (the website is down > but NNTP still works), that's why I have to start a new thread. :-( > > Am Tue, 30 Aug 2016 21:41:21 -0500 schrieb Sam Dyck: > > After reading through the changeset discussion, I discovered that one > > of my imports in Northern Manitoba made Worst of OSM. > > (http://worstofosm.tumblr.com/post/22180046353/dear- > > openstreetmap-isnt-it-strange-how-the). As someone who spends a some > > time amount of time in some of relatively unpopulated areas of Canada > > and makes an effort to check the quality of Canvec data (which is > > usually pretty good), I do agree that it is impossible to do > > everything to the same level of quality that we would provide in > > Toronto or Timmins or even small prairie towns. > > First of all, it is ok that an import takes a few years and therefore > creates ugly green rectancles on the map. If an import is "unavoidable" > :-), a manual import is the best thing that can be happen. But if someone > uploads a changeset without a manual review beforehand, he counteracts the > aim of a manual import: addind good data to OpenStreetMap. That's what I am > mainly fighting against. If a users uploads much more than 100 objects per > minute [1], you can be sure that he has not done any manual review. A > manual review by myself confirmed this these. I am fighting against such > changesets/users. > > A good imports must be reviewed *before* it is being uploaded. The review > contains: > - - Run JOSM validator, fix all warnings and errors. This includes all > warnings regarding validity of areas. (you can argue if all warnings about > "deprecated" tagging have to be fixed) > - - Compare the data with available imagery. Is the forest really a forest > or is another tag more appropiate? Right-click on a Bing tile at JOSM and > have a look how old/recent the imagery is. > - - Check if CanVec data fits to itself. > http://worstofosm.tumblr.com/post/22180046353/dear-openstreetmap-isnt- > it-strange-how-the > - - Check if there has been any other data before. If yes, adapt the > either the CanVec data or the old data. > https://wiki.openstreetmap.org/wiki/File:Import-Fails-Powerlines-Not-Ins > ide-Cutting.png > https://www.openstreetmap.org/way/439631732 >
Re: [Talk-ca] CanVec Reverts
Thank for contacting the Canadian community Michael, You provided us with a short but useful reminder of current rules we should apply when importing data (or even just making standard edits). However, I understand from your last paragraph that you will keep deleting changesets. I was hoping your email started a discussion on best practices that we could be put in place in our context since adjusting Canvec data to the latest rules is a daunting task. I rather feel it is an ultimatum. Do your future actions will apply to the imports made a few months, a few years ago, which are 'full of errors' and for which nobody seems to care? Are you going to check with concerned contributors (old/future imports) if they bother or not to see their work deleted before you do it? Furthermore, I hope you will not use you 100 objects per minute to decide whether or not you will delete a changeset. I think this threshold is value doesn't' apply (see below) Daniel About the100 objects threshold. From my experience, if I load a Canvec tile in JOSM, make all the necessary corrections and then import the result to OSM, I throw up to 25K objects to the database within five minutes. As far as I know, the timestamps attached to the changeset and to the objects is generated by the OSM database when receiving the data. The five minutes it takes to upload the data to the database (5K objects per minute) do not reflect the time I spent editing the data prior to the upload. -Original Message- From: Michael Reichert [mailto:naka...@gmx.net] Sent: Thursday, 1 September, 2016 01:39 To: talk-ca@openstreetmap.org Subject: [Talk-ca] CanVec Reverts -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, unfortunately posting via Gmane does not seem to work (the website is down but NNTP still works), that's why I have to start a new thread. :-( Am Tue, 30 Aug 2016 21:41:21 -0500 schrieb Sam Dyck: > After reading through the changeset discussion, I discovered that one > of my imports in Northern Manitoba made Worst of OSM. > (http://worstofosm.tumblr.com/post/22180046353/dear- > openstreetmap-isnt-it-strange-how-the). As someone who spends a some > time amount of time in some of relatively unpopulated areas of Canada > and makes an effort to check the quality of Canvec data (which is > usually pretty good), I do agree that it is impossible to do > everything to the same level of quality that we would provide in > Toronto or Timmins or even small prairie towns. First of all, it is ok that an import takes a few years and therefore creates ugly green rectancles on the map. If an import is "unavoidable" :-), a manual import is the best thing that can be happen. But if someone uploads a changeset without a manual review beforehand, he counteracts the aim of a manual import: addind good data to OpenStreetMap. That's what I am mainly fighting against. If a users uploads much more than 100 objects per minute [1], you can be sure that he has not done any manual review. A manual review by myself confirmed this these. I am fighting against such changesets/users. A good imports must be reviewed *before* it is being uploaded. The review contains: - - Run JOSM validator, fix all warnings and errors. This includes all warnings regarding validity of areas. (you can argue if all warnings about "deprecated" tagging have to be fixed) - - Compare the data with available imagery. Is the forest really a forest or is another tag more appropiate? Right-click on a Bing tile at JOSM and have a look how old/recent the imagery is. - - Check if CanVec data fits to itself. http://worstofosm.tumblr.com/post/22180046353/dear-openstreetmap-isnt- it-strange-how-the - - Check if there has been any other data before. If yes, adapt the either the CanVec data or the old data. https://wiki.openstreetmap.org/wiki/File:Import-Fails-Powerlines-Not-Ins ide-Cutting.png https://www.openstreetmap.org/way/439631732 - - Ways should not overlap with other ways if it is not necessary. The outer ring of a lake should also be inner member of the forest multipolygon. Maybe the program which created the OSM files should be imprved? - - Keep the history. https://wiki.openstreetmap.org/wiki/Good_practice#Keep_the_history If a tile has been imported without being checked manually and no post-upload fixes have been done (i.e. upload without any checks), I will not shrink from reverting it. If a tile has been uploaded to OSM without a review and if it has not been fixed within a month, it is worthless and can easily be reimported at a later time if someone has the time to check and fix it. For the future, I will abstain from reverting changesets which have been imported before September 1, 2016 and whose users are currently doing the fixes that should already have been done. But if I come across an imported tile of low quality which has not been touched for a few weeks and is full
Re: [Talk-ca] CanVec Reverts
See that's where I have an issue with your revert logic, if errors are mostly corrected and say theres only 1 or 2 warnings, you want to revert the whole thing which is very bad for: 1. The community and 2. The database. Let me explain: there are many hours that go into merging down CanVec ways and you want to undo all that work. Why is it bad for the database? It makes it bigger everytine you do stuff like that, an ID has to be allocated to every node, way and relation and by you "deleting" everything, they remain in the database as history which is bad as the full history planet file is already retardedly big(200+GB) and then when someone redoes the work you undid, new IDs are allocated which doubles the data size just for you being anal about perfection. Instead of reverting it for 1-2 warnings FIX IT! You dont have nice bing satelite imagery to reference(which is the case with most of northern Canada)? Too bad you want to be anal. Also this is a reminder that bing imagery can be 1. Really really really stale(10-15 years old as taking high res photos of trees doesnt return on investment) and 2. 1-20m offset. Instead of reverting the data that isnt perfect, why not spend your time correcting it instead of work being done: Canada isn't Germany. Ich wünsche einen guten Tag! On Sep 1, 2016 1:40 AM, "Michael Reichert"wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi, > > unfortunately posting via Gmane does not seem to work (the website is > down but NNTP still works), that's why I have to start a new thread. :-( > > Am Tue, 30 Aug 2016 21:41:21 -0500 schrieb Sam Dyck: > > After reading through the changeset discussion, I discovered that > > one of my imports in Northern Manitoba made Worst of OSM. > > (http://worstofosm.tumblr.com/post/22180046353/dear- > > openstreetmap-isnt-it-strange-how-the). As someone who spends a > > some time amount of time in some of relatively unpopulated areas of > > Canada and makes an effort to check the quality of Canvec data > > (which is usually pretty good), I do agree that it is impossible to > > do everything to the same level of quality that we would provide in > > Toronto or Timmins or even small prairie towns. > > First of all, it is ok that an import takes a few years and therefore > creates ugly green rectancles on the map. If an import is "unavoidable" > :-), a manual import is the best thing that can be happen. But if > someone uploads a changeset without a manual review beforehand, he > counteracts the aim of a manual import: addind good data to > OpenStreetMap. That's what I am mainly fighting against. If a users > uploads much more than 100 objects per minute [1], you can be sure that > he has not done any manual review. A manual review by myself confirmed > this these. I am fighting against such changesets/users. > > A good imports must be reviewed *before* it is being uploaded. The > review contains: > - - Run JOSM validator, fix all warnings and errors. This includes all > warnings regarding validity of areas. (you can argue if all warnings > about "deprecated" tagging have to be fixed) > - - Compare the data with available imagery. Is the forest really a forest > or is another tag more appropiate? Right-click on a Bing tile at JOSM > and have a look how old/recent the imagery is. > - - Check if CanVec data fits to itself. > http://worstofosm.tumblr.com/post/22180046353/dear-openstreetmap-isnt- > it-strange-how-the > - - Check if there has been any other data before. If yes, adapt the > either the CanVec data or the old data. > https://wiki.openstreetmap.org/wiki/File:Import-Fails-Powerlines-Not-Ins > ide-Cutting.png > https://www.openstreetmap.org/way/439631732 > - - Ways should not overlap with other ways if it is not necessary. The > outer ring of a lake should also be inner member of the forest > multipolygon. Maybe the program which created the OSM files should be > imprved? > - - Keep the history. > https://wiki.openstreetmap.org/wiki/Good_practice#Keep_the_history > > If a tile has been imported without being checked manually and no > post-upload fixes have been done (i.e. upload without any checks), I > will not shrink from reverting it. If a tile has been uploaded to OSM > without a review and if it has not been fixed within a month, it is > worthless and can easily be reimported at a later time if someone has > the time to check and fix it. > > For the future, I will abstain from reverting changesets which have been > imported before September 1, 2016 and whose users are currently doing > the fixes that should already have been done. But if I come across an > imported tile of low quality which has not been touched for a few weeks > and is full of errors, it is just a question of time until it is reverte > d. > > Best regards > > Michael > > > > [1] I had a look on a few of my changesets which added a large number of > buildings to OSM. The fastest changeset contained about 60 objects per > minute and was full
[Talk-ca] CanVec Reverts
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, unfortunately posting via Gmane does not seem to work (the website is down but NNTP still works), that's why I have to start a new thread. :-( Am Tue, 30 Aug 2016 21:41:21 -0500 schrieb Sam Dyck: > After reading through the changeset discussion, I discovered that > one of my imports in Northern Manitoba made Worst of OSM. > (http://worstofosm.tumblr.com/post/22180046353/dear- > openstreetmap-isnt-it-strange-how-the). As someone who spends a > some time amount of time in some of relatively unpopulated areas of > Canada and makes an effort to check the quality of Canvec data > (which is usually pretty good), I do agree that it is impossible to > do everything to the same level of quality that we would provide in > Toronto or Timmins or even small prairie towns. First of all, it is ok that an import takes a few years and therefore creates ugly green rectancles on the map. If an import is "unavoidable" :-), a manual import is the best thing that can be happen. But if someone uploads a changeset without a manual review beforehand, he counteracts the aim of a manual import: addind good data to OpenStreetMap. That's what I am mainly fighting against. If a users uploads much more than 100 objects per minute [1], you can be sure that he has not done any manual review. A manual review by myself confirmed this these. I am fighting against such changesets/users. A good imports must be reviewed *before* it is being uploaded. The review contains: - - Run JOSM validator, fix all warnings and errors. This includes all warnings regarding validity of areas. (you can argue if all warnings about "deprecated" tagging have to be fixed) - - Compare the data with available imagery. Is the forest really a forest or is another tag more appropiate? Right-click on a Bing tile at JOSM and have a look how old/recent the imagery is. - - Check if CanVec data fits to itself. http://worstofosm.tumblr.com/post/22180046353/dear-openstreetmap-isnt- it-strange-how-the - - Check if there has been any other data before. If yes, adapt the either the CanVec data or the old data. https://wiki.openstreetmap.org/wiki/File:Import-Fails-Powerlines-Not-Ins ide-Cutting.png https://www.openstreetmap.org/way/439631732 - - Ways should not overlap with other ways if it is not necessary. The outer ring of a lake should also be inner member of the forest multipolygon. Maybe the program which created the OSM files should be imprved? - - Keep the history. https://wiki.openstreetmap.org/wiki/Good_practice#Keep_the_history If a tile has been imported without being checked manually and no post-upload fixes have been done (i.e. upload without any checks), I will not shrink from reverting it. If a tile has been uploaded to OSM without a review and if it has not been fixed within a month, it is worthless and can easily be reimported at a later time if someone has the time to check and fix it. For the future, I will abstain from reverting changesets which have been imported before September 1, 2016 and whose users are currently doing the fixes that should already have been done. But if I come across an imported tile of low quality which has not been touched for a few weeks and is full of errors, it is just a question of time until it is reverte d. Best regards Michael [1] I had a look on a few of my changesets which added a large number of buildings to OSM. The fastest changeset contained about 60 objects per minute and was full of missing buildings as I later detected while collecting the housenumbers and usage of the buildings. - -- Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten ausgenommen) I prefer GPG encryption of emails. (does not apply on mailing lists) -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCAAGBQJXx77+AAoJEB87G9rMCMyI95YQAJyCMY/Qtnqu4C3MpLPrrwTH vN2aVBNoKHL+i6r5VBTPFy4JhcacjbAZSseMCbmQHo6CSdPgVk9Jvnk/Keh3ihH6 r//EqwqRnSPPE+JIBktW0DG50jzcogWun3UQmOyA/blRfYEZaIDhjRDjMcivBWvs x8EGZsuX/mraX74ucTbfhy13Xoy4M4Yjf4ibNS7ZmJKlT4Zj8HDwlPKRzFxZ6iWG JGfibU4wxxvJlQDWqjrVRN6zacbt8SKh9sHU3M88kNRhM+eido/rYY9LiKBFdO21 TBGM/XkvxcqM7F9u9uC03caDFi0cTb7PLZgv90QhXTi2DuFobfHf3sc1czq8lGeB Wa8ZZqRI6Y9/SV06P/wydA48caKeeO3OiiuH8UXzEZJPauUhLjpEVxY1OixkgNkC GR79KbVcx0AyFK66Jnrdn2pqa2HotJ2rM6RLSi68mMOYPbHcUvujcb7XQW5dvubF L3TamnVs8u1qiifpfTCAp/AJFxd/9UDnGi2VsjSHrIPdZgJaCAcHmhiUSXkcZa55 hjfL+4b+itj48PRcfJRzXTRE3I9Q7oAyMkbwMKVFvSOe9GwgUNw5nvspWvPUriUo pDoDHFJt3k4RE6hHVhsb0+L/OyVr6NFpjex2aoEbX0990Gvi+G6uabkNAOn4o0Ub nAQhtQWnI5dlwMWhcYOH =vPJv -END PGP SIGNATURE- ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca