Re: [Talk-ee] verbatium buiding import discussion
I don't know how to use osmose-qa, sorry. Среда, 25 сентября 2019, 8:08 +03:00 от "Jaak Laineste" : > Would you consider using osmose-qa here? It would require one-time creating > connector for maa-amet buildings shp, and this will be useful in any case, for > whole country and for long time, permanently even. > > Then there are two ways to apply it for Tallinn: (a) remove all verbatium and > then it shows all deleted buildings as missing or (b) softer - no delete, > shows latest maaamet ones as just more uptodate and users can click through > each. Maybe it has some more bulk updating also. I’d start with the soft one, > helps to precheck the changes also, even if you end up bulk delete+upload. > > Creating and deploy proper connector which identifies both maaamet real > updates and verbatium properly may require some learning and sweat. > > Jaak > (Sent from mobile) > > > On 24 Sep 2019, at 23:46, SviMik wrote: > > > > http://svimik.com/verbatiumimportmap1.png > > Here's the screenshot. Not precise, but gives some idea. > > 40903 buildings is a bit too much for JOSM. > > > > > > Вторник, 24 сентября 2019, 22:08 +03:00 от "Jaak Laineste" > : > >> > >> These txt reports with IDs are not too visual, can you create any > renderable > >> file for josm (.osm for example) or screenshot to show the community what > it > >> really means? Are the building under question random or somehow in specific > >> areas etc. I dont know if there is any "test osm server” for such > experiments > >> nowadays somewhere, probably not. > >> > >> Jaak > >> > >>> On 24 Sep 2019, at 07:50, SviMik wrote: > >>> > >>> I have analyzed the largest verbatium's import (changesets 591093, 579407, > >> 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, > >> 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers > >> 86.7% of all his edits. > >>> The import was covering Tallinn, Saue and Maardu. Turned out verbatium > >> wasn't particulary active outside Harjumaa. > >>> There were total 67813 ways imported, 9032 of them are already deleted by > >> other editors, and 17878 were changed either in geometry or tags* > department > >> (15529 has a geometry change, and 5077 has a tag change), which leaves > 40903 > >> buildings for deletion. > >>> * I did not count the following tags because they are part of my imports > >> anyway: addr:city, addr:country, addr:housenumber, addr:street, > maaamet:ETAK, > >> maaamet:orig_tunnus, source, source:addr, addr2:* > >>> * If some change was reverted I do not count it as a change, because I > only > >> compare the first and the last version > >>> > >>> Here is the list of ways for deletion: > >>> http://svimik.com/verbatium_import1_ways_unchanged1.txt > >>> > >>> Here is the full report for all the 67813 ways: > >>> http://svimik.com/verbatium_import1_ways1.csv > >>> > >>> Full list of his changesets: > >>> http://svimik.com/verbatium_changesets.xls > >>> > >>> Currently, the bbox of his changesets has 91324 buildings, which means... > >> We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be > >> interesting. Should we split it by 10k for example? Or just start with > Maardu > >> and see what happens? > >>> > >>> > >>> Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" > >> : > Hoi, > > Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju > >> ja > kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. > > Jaak > > p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), > >> need > on pigem müra kui info kaardil. > > > > > On 14 Sep 2019, at 15:34, SviMik via Talk-ee > wrote: > > > > Hi everyone! > > > > I'd like to discuss the import made by verbatium in 2008: > > https://www.openstreetmap.org/changeset/569055 > > (...and other similar changesets) > > > > There are two issues with that import: > > 1. Unknown data source with unknown license (probably it was a Garmin > map > judging by the Type=0x13 tag) > > 2. Poor quality of the data. If you see a building distorted like this: > http://svimik.com/verbatiumimport1.png > > - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 > >> we > have much better options. > > > > What can be done: > > 1. Remove all buildings which geometry and tags were not edited since > the > initial import. For the tags the following exceptions can be made because > >> they > were automatic edits: > > - User xybot has fixed the tag typo (buildung=yes) in the initial import > >> and > added its own tag (created_by=xybot) > > - User juhanjuku has removed the Type=0x13 and created_by=xybot tags > > - User SviMik_import has imported the address tags to these buildings > from > the Maa-amet database (nothing that can't be imported again) > > 2.
Re: [Talk-ee] verbatium buiding import discussion
All these edits are automatic. juhanjuku and kaupov were mass-removing spam tag "Type=0x13" from verbatium's import, I don't consider it an 'edit'. Среда, 25 сентября 2019, 8:35 +03:00 от "Jaak Laineste" : > > Hi, > > When I did Tartu bulk delete-replace with the city gov data long time ago, > then I contacted all the previous editors in the area and asked their > permission. As number of existing data was small, then they were ok. > > Here most buildings seem to have some manual or semi-manual edits after > import, I would group the edits by the involved users (juhanjuku, kaupov seem > to be popular), and ask their permission before basically deleting their > efforts. Where you dont get permission better use soft approach there: just > mark changes for manual checks by community. > > > Jaak > > > > > On 24 Sep 2019, at 07:50, SviMik wrote: > > > > I have analyzed the largest verbatium's import (changesets 591093, 579407, > 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, > 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers > 86.7% of all his edits. > > The import was covering Tallinn, Saue and Maardu. Turned out verbatium > wasn't particulary active outside Harjumaa. > > There were total 67813 ways imported, 9032 of them are already deleted by > other editors, and 17878 were changed either in geometry or tags* department > (15529 has a geometry change, and 5077 has a tag change), which leaves 40903 > buildings for deletion. > > * I did not count the following tags because they are part of my imports > anyway: addr:city, addr:country, addr:housenumber, addr:street, maaamet:ETAK, > maaamet:orig_tunnus, source, source:addr, addr2:* > > * If some change was reverted I do not count it as a change, because I only > compare the first and the last version > > > > Here is the list of ways for deletion: > > http://svimik.com/verbatium_import1_ways_unchanged1.txt > > > > Here is the full report for all the 67813 ways: > > http://svimik.com/verbatium_import1_ways1.csv > > > > Full list of his changesets: > > http://svimik.com/verbatium_changesets.xls > > > > Currently, the bbox of his changesets has 91324 buildings, which means... > We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be > interesting. Should we split it by 10k for example? Or just start with Maardu > and see what happens? > > > > > > Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" > : > >> Hoi, > >> > >> Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju > ja > >> kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. > >> > >> Jaak > >> > >> p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), > need > >> on pigem müra kui info kaardil. > >> > >> > >> > >>> On 14 Sep 2019, at 15:34, SviMik via Talk-ee > >> wrote: > >>> > >>> Hi everyone! > >>> > >>> I'd like to discuss the import made by verbatium in 2008: > >>> https://www.openstreetmap.org/changeset/569055 > >>> (...and other similar changesets) > >>> > >>> There are two issues with that import: > >>> 1. Unknown data source with unknown license (probably it was a Garmin map > >> judging by the Type=0x13 tag) > >>> 2. Poor quality of the data. If you see a building distorted like this: > >> http://svimik.com/verbatiumimport1.png > >>> - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 > we > >> have much better options. > >>> > >>> What can be done: > >>> 1. Remove all buildings which geometry and tags were not edited since the > >> initial import. For the tags the following exceptions can be made because > they > >> were automatic edits: > >>> - User xybot has fixed the tag typo (buildung=yes) in the initial import > and > >> added its own tag (created_by=xybot) > >>> - User juhanjuku has removed the Type=0x13 and created_by=xybot tags > >>> - User SviMik_import has imported the address tags to these buildings from > >> the Maa-amet database (nothing that can't be imported again) > >>> 2. Proceed with the Maa-amet building import as usual > >>> > >>> It will solve: > >>> 1. The license issue (if there is any) > >>> 2. The quality issue (if you agree there is an issue) > >>> 3. Will update the map in general, for example the demolished buildings > will > >> be removed from OSM. > >>> > >>> For buildings which geometry was changed by other contributors after the > >> initial import - we can assume both license and quality issues were solved > >> since they no longer contain the imported geometry. I know it's a grey > field, > >> and I'm not sure it works like that, but at least these buildings do have > some > >> excuse to stay. > >>> > >>> For buildings which geometry was NOT changed, but some POI tags were added > - > >> let them stay for now and discuss it later if needed. I suspect it will be > a > >> rare case, but the exact number is unknown right now. > >>> > >>> Questions: > >>> 1. Has anyone else digged into
Re: [Talk-ee] verbatium buiding import discussion
Hi, When I did Tartu bulk delete-replace with the city gov data long time ago, then I contacted all the previous editors in the area and asked their permission. As number of existing data was small, then they were ok. Here most buildings seem to have some manual or semi-manual edits after import, I would group the edits by the involved users (juhanjuku, kaupov seem to be popular), and ask their permission before basically deleting their efforts. Where you dont get permission better use soft approach there: just mark changes for manual checks by community. Jaak > On 24 Sep 2019, at 07:50, SviMik wrote: > > I have analyzed the largest verbatium's import (changesets 591093, 579407, > 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, > 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers > 86.7% of all his edits. > The import was covering Tallinn, Saue and Maardu. Turned out verbatium wasn't > particulary active outside Harjumaa. > There were total 67813 ways imported, 9032 of them are already deleted by > other editors, and 17878 were changed either in geometry or tags* department > (15529 has a geometry change, and 5077 has a tag change), which leaves 40903 > buildings for deletion. > * I did not count the following tags because they are part of my imports > anyway: addr:city, addr:country, addr:housenumber, addr:street, maaamet:ETAK, > maaamet:orig_tunnus, source, source:addr, addr2:* > * If some change was reverted I do not count it as a change, because I only > compare the first and the last version > > Here is the list of ways for deletion: > http://svimik.com/verbatium_import1_ways_unchanged1.txt > > Here is the full report for all the 67813 ways: > http://svimik.com/verbatium_import1_ways1.csv > > Full list of his changesets: > http://svimik.com/verbatium_changesets.xls > > Currently, the bbox of his changesets has 91324 buildings, which means... > We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be > interesting. Should we split it by 10k for example? Or just start with Maardu > and see what happens? > > > Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" > : >> Hoi, >> >> Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju ja >> kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. >> >> Jaak >> >> p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), need >> on pigem müra kui info kaardil. >> >> >> >>> On 14 Sep 2019, at 15:34, SviMik via Talk-ee >> wrote: >>> >>> Hi everyone! >>> >>> I'd like to discuss the import made by verbatium in 2008: >>> https://www.openstreetmap.org/changeset/569055 >>> (...and other similar changesets) >>> >>> There are two issues with that import: >>> 1. Unknown data source with unknown license (probably it was a Garmin map >> judging by the Type=0x13 tag) >>> 2. Poor quality of the data. If you see a building distorted like this: >> http://svimik.com/verbatiumimport1.png >>> - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 we >> have much better options. >>> >>> What can be done: >>> 1. Remove all buildings which geometry and tags were not edited since the >> initial import. For the tags the following exceptions can be made because >> they >> were automatic edits: >>> - User xybot has fixed the tag typo (buildung=yes) in the initial import and >> added its own tag (created_by=xybot) >>> - User juhanjuku has removed the Type=0x13 and created_by=xybot tags >>> - User SviMik_import has imported the address tags to these buildings from >> the Maa-amet database (nothing that can't be imported again) >>> 2. Proceed with the Maa-amet building import as usual >>> >>> It will solve: >>> 1. The license issue (if there is any) >>> 2. The quality issue (if you agree there is an issue) >>> 3. Will update the map in general, for example the demolished buildings will >> be removed from OSM. >>> >>> For buildings which geometry was changed by other contributors after the >> initial import - we can assume both license and quality issues were solved >> since they no longer contain the imported geometry. I know it's a grey field, >> and I'm not sure it works like that, but at least these buildings do have >> some >> excuse to stay. >>> >>> For buildings which geometry was NOT changed, but some POI tags were added - >> let them stay for now and discuss it later if needed. I suspect it will be a >> rare case, but the exact number is unknown right now. >>> >>> Questions: >>> 1. Has anyone else digged into the issue, maybe asked verbatium himself? >>> 2. Can anyone confirm that the import indeed has the license problem? >>> 3. Is the proposed plan good? (in case if you agree that it needs to be >> fixed) >>> >>> -- >>> SviMik >>> ___ >>> Talk-ee mailing list >>> Talk-ee@openstreetmap.org >>> https://lists.openstreetmap.org/listinfo/talk-ee >> >>
Re: [Talk-ee] verbatium buiding import discussion
Would you consider using osmose-qa here? It would require one-time creating connector for maa-amet buildings shp, and this will be useful in any case, for whole country and for long time, permanently even. Then there are two ways to apply it for Tallinn: (a) remove all verbatium and then it shows all deleted buildings as missing or (b) softer - no delete, shows latest maaamet ones as just more uptodate and users can click through each. Maybe it has some more bulk updating also. I’d start with the soft one, helps to precheck the changes also, even if you end up bulk delete+upload. Creating and deploy proper connector which identifies both maaamet real updates and verbatium properly may require some learning and sweat. Jaak (Sent from mobile) > On 24 Sep 2019, at 23:46, SviMik wrote: > > http://svimik.com/verbatiumimportmap1.png > Here's the screenshot. Not precise, but gives some idea. > 40903 buildings is a bit too much for JOSM. > > > Вторник, 24 сентября 2019, 22:08 +03:00 от "Jaak Laineste" : >> >> These txt reports with IDs are not too visual, can you create any renderable >> file for josm (.osm for example) or screenshot to show the community what it >> really means? Are the building under question random or somehow in specific >> areas etc. I dont know if there is any "test osm server” for such >> experiments >> nowadays somewhere, probably not. >> >> Jaak >> >>> On 24 Sep 2019, at 07:50, SviMik wrote: >>> >>> I have analyzed the largest verbatium's import (changesets 591093, 579407, >> 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, >> 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers >> 86.7% of all his edits. >>> The import was covering Tallinn, Saue and Maardu. Turned out verbatium >> wasn't particulary active outside Harjumaa. >>> There were total 67813 ways imported, 9032 of them are already deleted by >> other editors, and 17878 were changed either in geometry or tags* department >> (15529 has a geometry change, and 5077 has a tag change), which leaves 40903 >> buildings for deletion. >>> * I did not count the following tags because they are part of my imports >> anyway: addr:city, addr:country, addr:housenumber, addr:street, maaamet:ETAK, >> maaamet:orig_tunnus, source, source:addr, addr2:* >>> * If some change was reverted I do not count it as a change, because I only >> compare the first and the last version >>> >>> Here is the list of ways for deletion: >>> http://svimik.com/verbatium_import1_ways_unchanged1.txt >>> >>> Here is the full report for all the 67813 ways: >>> http://svimik.com/verbatium_import1_ways1.csv >>> >>> Full list of his changesets: >>> http://svimik.com/verbatium_changesets.xls >>> >>> Currently, the bbox of his changesets has 91324 buildings, which means... >> We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be >> interesting. Should we split it by 10k for example? Or just start with Maardu >> and see what happens? >>> >>> >>> Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" >> : Hoi, Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju >> ja kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. Jaak p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), >> need on pigem müra kui info kaardil. > On 14 Sep 2019, at 15:34, SviMik via Talk-ee wrote: > > Hi everyone! > > I'd like to discuss the import made by verbatium in 2008: > https://www.openstreetmap.org/changeset/569055 > (...and other similar changesets) > > There are two issues with that import: > 1. Unknown data source with unknown license (probably it was a Garmin map judging by the Type=0x13 tag) > 2. Poor quality of the data. If you see a building distorted like this: http://svimik.com/verbatiumimport1.png > - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 >> we have much better options. > > What can be done: > 1. Remove all buildings which geometry and tags were not edited since the initial import. For the tags the following exceptions can be made because >> they were automatic edits: > - User xybot has fixed the tag typo (buildung=yes) in the initial import >> and added its own tag (created_by=xybot) > - User juhanjuku has removed the Type=0x13 and created_by=xybot tags > - User SviMik_import has imported the address tags to these buildings from the Maa-amet database (nothing that can't be imported again) > 2. Proceed with the Maa-amet building import as usual > > It will solve: > 1. The license issue (if there is any) > 2. The quality issue (if you agree there is an issue) > 3. Will update the map in general, for example the demolished buildings >> will be removed from OSM. > > For buildings which geometry
Re: [Talk-ee] verbatium buiding import discussion
Hi, osm test server exists complete with rendering. https://wiki.openstreetmap.org/wiki/Sandbox_for_editing Regards, Mihkel On Tue, Sep 24, 2019 at 10:08 PM Jaak Laineste wrote: > > > These txt reports with IDs are not too visual, can you create any renderable > file for josm (.osm for example) or screenshot to show the community what it > really means? Are the building under question random or somehow in specific > areas etc. I dont know if there is any "test osm server” for such > experiments nowadays somewhere, probably not. > > Jaak > > > On 24 Sep 2019, at 07:50, SviMik wrote: > > > > I have analyzed the largest verbatium's import (changesets 591093, 579407, > > 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, > > 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers > > 86.7% of all his edits. > > The import was covering Tallinn, Saue and Maardu. Turned out verbatium > > wasn't particulary active outside Harjumaa. > > There were total 67813 ways imported, 9032 of them are already deleted by > > other editors, and 17878 were changed either in geometry or tags* > > department (15529 has a geometry change, and 5077 has a tag change), which > > leaves 40903 buildings for deletion. > > * I did not count the following tags because they are part of my imports > > anyway: addr:city, addr:country, addr:housenumber, addr:street, > > maaamet:ETAK, maaamet:orig_tunnus, source, source:addr, addr2:* > > * If some change was reverted I do not count it as a change, because I only > > compare the first and the last version > > > > Here is the list of ways for deletion: > > http://svimik.com/verbatium_import1_ways_unchanged1.txt > > > > Here is the full report for all the 67813 ways: > > http://svimik.com/verbatium_import1_ways1.csv > > > > Full list of his changesets: > > http://svimik.com/verbatium_changesets.xls > > > > Currently, the bbox of his changesets has 91324 buildings, which means... > > We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be > > interesting. Should we split it by 10k for example? Or just start with > > Maardu and see what happens? > > > > > > Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" > > : > >> Hoi, > >> > >> Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju > >> ja > >> kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. > >> > >> Jaak > >> > >> p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), > >> need > >> on pigem müra kui info kaardil. > >> > >> > >> > >>> On 14 Sep 2019, at 15:34, SviMik via Talk-ee > >> wrote: > >>> > >>> Hi everyone! > >>> > >>> I'd like to discuss the import made by verbatium in 2008: > >>> https://www.openstreetmap.org/changeset/569055 > >>> (...and other similar changesets) > >>> > >>> There are two issues with that import: > >>> 1. Unknown data source with unknown license (probably it was a Garmin map > >> judging by the Type=0x13 tag) > >>> 2. Poor quality of the data. If you see a building distorted like this: > >> http://svimik.com/verbatiumimport1.png > >>> - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 > >>> we > >> have much better options. > >>> > >>> What can be done: > >>> 1. Remove all buildings which geometry and tags were not edited since the > >> initial import. For the tags the following exceptions can be made because > >> they > >> were automatic edits: > >>> - User xybot has fixed the tag typo (buildung=yes) in the initial import > >>> and > >> added its own tag (created_by=xybot) > >>> - User juhanjuku has removed the Type=0x13 and created_by=xybot tags > >>> - User SviMik_import has imported the address tags to these buildings from > >> the Maa-amet database (nothing that can't be imported again) > >>> 2. Proceed with the Maa-amet building import as usual > >>> > >>> It will solve: > >>> 1. The license issue (if there is any) > >>> 2. The quality issue (if you agree there is an issue) > >>> 3. Will update the map in general, for example the demolished buildings > >>> will > >> be removed from OSM. > >>> > >>> For buildings which geometry was changed by other contributors after the > >> initial import - we can assume both license and quality issues were solved > >> since they no longer contain the imported geometry. I know it's a grey > >> field, > >> and I'm not sure it works like that, but at least these buildings do have > >> some > >> excuse to stay. > >>> > >>> For buildings which geometry was NOT changed, but some POI tags were > >>> added - > >> let them stay for now and discuss it later if needed. I suspect it will be > >> a > >> rare case, but the exact number is unknown right now. > >>> > >>> Questions: > >>> 1. Has anyone else digged into the issue, maybe asked verbatium himself? > >>> 2. Can anyone confirm that the import indeed has the license problem? > >>> 3. Is the proposed plan good? (in case if you agree that it needs to be > >>
Re: [Talk-ee] verbatium buiding import discussion
http://svimik.com/verbatiumimportmap1.png Here's the screenshot. Not precise, but gives some idea. 40903 buildings is a bit too much for JOSM. Вторник, 24 сентября 2019, 22:08 +03:00 от "Jaak Laineste" : > > These txt reports with IDs are not too visual, can you create any renderable > file for josm (.osm for example) or screenshot to show the community what it > really means? Are the building under question random or somehow in specific > areas etc. I dont know if there is any "test osm server” for such experiments > nowadays somewhere, probably not. > > Jaak > > > On 24 Sep 2019, at 07:50, SviMik wrote: > > > > I have analyzed the largest verbatium's import (changesets 591093, 579407, > 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, > 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers > 86.7% of all his edits. > > The import was covering Tallinn, Saue and Maardu. Turned out verbatium > wasn't particulary active outside Harjumaa. > > There were total 67813 ways imported, 9032 of them are already deleted by > other editors, and 17878 were changed either in geometry or tags* department > (15529 has a geometry change, and 5077 has a tag change), which leaves 40903 > buildings for deletion. > > * I did not count the following tags because they are part of my imports > anyway: addr:city, addr:country, addr:housenumber, addr:street, maaamet:ETAK, > maaamet:orig_tunnus, source, source:addr, addr2:* > > * If some change was reverted I do not count it as a change, because I only > compare the first and the last version > > > > Here is the list of ways for deletion: > > http://svimik.com/verbatium_import1_ways_unchanged1.txt > > > > Here is the full report for all the 67813 ways: > > http://svimik.com/verbatium_import1_ways1.csv > > > > Full list of his changesets: > > http://svimik.com/verbatium_changesets.xls > > > > Currently, the bbox of his changesets has 91324 buildings, which means... > We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be > interesting. Should we split it by 10k for example? Or just start with Maardu > and see what happens? > > > > > > Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" > : > >> Hoi, > >> > >> Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju > ja > >> kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. > >> > >> Jaak > >> > >> p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), > need > >> on pigem müra kui info kaardil. > >> > >> > >> > >>> On 14 Sep 2019, at 15:34, SviMik via Talk-ee > >> wrote: > >>> > >>> Hi everyone! > >>> > >>> I'd like to discuss the import made by verbatium in 2008: > >>> https://www.openstreetmap.org/changeset/569055 > >>> (...and other similar changesets) > >>> > >>> There are two issues with that import: > >>> 1. Unknown data source with unknown license (probably it was a Garmin map > >> judging by the Type=0x13 tag) > >>> 2. Poor quality of the data. If you see a building distorted like this: > >> http://svimik.com/verbatiumimport1.png > >>> - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 > we > >> have much better options. > >>> > >>> What can be done: > >>> 1. Remove all buildings which geometry and tags were not edited since the > >> initial import. For the tags the following exceptions can be made because > they > >> were automatic edits: > >>> - User xybot has fixed the tag typo (buildung=yes) in the initial import > and > >> added its own tag (created_by=xybot) > >>> - User juhanjuku has removed the Type=0x13 and created_by=xybot tags > >>> - User SviMik_import has imported the address tags to these buildings from > >> the Maa-amet database (nothing that can't be imported again) > >>> 2. Proceed with the Maa-amet building import as usual > >>> > >>> It will solve: > >>> 1. The license issue (if there is any) > >>> 2. The quality issue (if you agree there is an issue) > >>> 3. Will update the map in general, for example the demolished buildings > will > >> be removed from OSM. > >>> > >>> For buildings which geometry was changed by other contributors after the > >> initial import - we can assume both license and quality issues were solved > >> since they no longer contain the imported geometry. I know it's a grey > field, > >> and I'm not sure it works like that, but at least these buildings do have > some > >> excuse to stay. > >>> > >>> For buildings which geometry was NOT changed, but some POI tags were added > - > >> let them stay for now and discuss it later if needed. I suspect it will be > a > >> rare case, but the exact number is unknown right now. > >>> > >>> Questions: > >>> 1. Has anyone else digged into the issue, maybe asked verbatium himself? > >>> 2. Can anyone confirm that the import indeed has the license problem? > >>> 3. Is the proposed plan good? (in case if you agree that it needs to be > >> fixed) > >>> > >>> -- > >>> SviMik > >>>
Re: [Talk-ee] verbatium buiding import discussion
These txt reports with IDs are not too visual, can you create any renderable file for josm (.osm for example) or screenshot to show the community what it really means? Are the building under question random or somehow in specific areas etc. I dont know if there is any "test osm server” for such experiments nowadays somewhere, probably not. Jaak > On 24 Sep 2019, at 07:50, SviMik wrote: > > I have analyzed the largest verbatium's import (changesets 591093, 579407, > 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, > 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers > 86.7% of all his edits. > The import was covering Tallinn, Saue and Maardu. Turned out verbatium wasn't > particulary active outside Harjumaa. > There were total 67813 ways imported, 9032 of them are already deleted by > other editors, and 17878 were changed either in geometry or tags* department > (15529 has a geometry change, and 5077 has a tag change), which leaves 40903 > buildings for deletion. > * I did not count the following tags because they are part of my imports > anyway: addr:city, addr:country, addr:housenumber, addr:street, maaamet:ETAK, > maaamet:orig_tunnus, source, source:addr, addr2:* > * If some change was reverted I do not count it as a change, because I only > compare the first and the last version > > Here is the list of ways for deletion: > http://svimik.com/verbatium_import1_ways_unchanged1.txt > > Here is the full report for all the 67813 ways: > http://svimik.com/verbatium_import1_ways1.csv > > Full list of his changesets: > http://svimik.com/verbatium_changesets.xls > > Currently, the bbox of his changesets has 91324 buildings, which means... > We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be > interesting. Should we split it by 10k for example? Or just start with Maardu > and see what happens? > > > Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" > : >> Hoi, >> >> Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju ja >> kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. >> >> Jaak >> >> p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), need >> on pigem müra kui info kaardil. >> >> >> >>> On 14 Sep 2019, at 15:34, SviMik via Talk-ee >> wrote: >>> >>> Hi everyone! >>> >>> I'd like to discuss the import made by verbatium in 2008: >>> https://www.openstreetmap.org/changeset/569055 >>> (...and other similar changesets) >>> >>> There are two issues with that import: >>> 1. Unknown data source with unknown license (probably it was a Garmin map >> judging by the Type=0x13 tag) >>> 2. Poor quality of the data. If you see a building distorted like this: >> http://svimik.com/verbatiumimport1.png >>> - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 we >> have much better options. >>> >>> What can be done: >>> 1. Remove all buildings which geometry and tags were not edited since the >> initial import. For the tags the following exceptions can be made because >> they >> were automatic edits: >>> - User xybot has fixed the tag typo (buildung=yes) in the initial import and >> added its own tag (created_by=xybot) >>> - User juhanjuku has removed the Type=0x13 and created_by=xybot tags >>> - User SviMik_import has imported the address tags to these buildings from >> the Maa-amet database (nothing that can't be imported again) >>> 2. Proceed with the Maa-amet building import as usual >>> >>> It will solve: >>> 1. The license issue (if there is any) >>> 2. The quality issue (if you agree there is an issue) >>> 3. Will update the map in general, for example the demolished buildings will >> be removed from OSM. >>> >>> For buildings which geometry was changed by other contributors after the >> initial import - we can assume both license and quality issues were solved >> since they no longer contain the imported geometry. I know it's a grey field, >> and I'm not sure it works like that, but at least these buildings do have >> some >> excuse to stay. >>> >>> For buildings which geometry was NOT changed, but some POI tags were added - >> let them stay for now and discuss it later if needed. I suspect it will be a >> rare case, but the exact number is unknown right now. >>> >>> Questions: >>> 1. Has anyone else digged into the issue, maybe asked verbatium himself? >>> 2. Can anyone confirm that the import indeed has the license problem? >>> 3. Is the proposed plan good? (in case if you agree that it needs to be >> fixed) >>> >>> -- >>> SviMik >>> ___ >>> Talk-ee mailing list >>> Talk-ee@openstreetmap.org >>> https://lists.openstreetmap.org/listinfo/talk-ee >> >> > > > -- > Svjatoslav Mikhailov ___ Talk-ee mailing list Talk-ee@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ee
Re: [Talk-ee] verbatium buiding import discussion
I have analyzed the largest verbatium's import (changesets 591093, 579407, 572107, 569277, 569055 (ways) + 561094, 559707, 558636, 558056, 557568, 557358, 557193, 556899 (nodes only)), which was made in 2008. That covers 86.7% of all his edits. The import was covering Tallinn, Saue and Maardu. Turned out verbatium wasn't particulary active outside Harjumaa. There were total 67813 ways imported, 9032 of them are already deleted by other editors, and 17878 were changed either in geometry or tags* department (15529 has a geometry change, and 5077 has a tag change), which leaves 40903 buildings for deletion. * I did not count the following tags because they are part of my imports anyway: addr:city, addr:country, addr:housenumber, addr:street, maaamet:ETAK, maaamet:orig_tunnus, source, source:addr, addr2:* * If some change was reverted I do not count it as a change, because I only compare the first and the last version Here is the list of ways for deletion: http://svimik.com/verbatium_import1_ways_unchanged1.txt Here is the full report for all the 67813 ways: http://svimik.com/verbatium_import1_ways1.csv Full list of his changesets: http://svimik.com/verbatium_changesets.xls Currently, the bbox of his changesets has 91324 buildings, which means... We're gonna delete 44.79% of Saue-Tallinn-Maardu buildings. That gonna be interesting. Should we split it by 10k for example? Or just start with Maardu and see what happens? Воскресенье, 15 сентября 2019, 9:42 +03:00 от "Jaak Laineste" : > Hoi, > > Jah, need peaks kustutama. Enne võiks teha muudatuse analüüsi - kui palju ja > kus kustutataks, ega mõned linnad kohe väga tühjaks ei jää. > > Jaak > > p.s. sama asi ka corine impordi osade tag-idega, näiteks põllud (field), need > on pigem müra kui info kaardil. > > > > > On 14 Sep 2019, at 15:34, SviMik via Talk-ee > wrote: > > > > Hi everyone! > > > > I'd like to discuss the import made by verbatium in 2008: > > https://www.openstreetmap.org/changeset/569055 > > (...and other similar changesets) > > > > There are two issues with that import: > > 1. Unknown data source with unknown license (probably it was a Garmin map > judging by the Type=0x13 tag) > > 2. Poor quality of the data. If you see a building distorted like this: > http://svimik.com/verbatiumimport1.png > > - you can be sure it's verbatium's. Maybe it was OK in 2008, but in 2019 we > have much better options. > > > > What can be done: > > 1. Remove all buildings which geometry and tags were not edited since the > initial import. For the tags the following exceptions can be made because they > were automatic edits: > > - User xybot has fixed the tag typo (buildung=yes) in the initial import and > added its own tag (created_by=xybot) > > - User juhanjuku has removed the Type=0x13 and created_by=xybot tags > > - User SviMik_import has imported the address tags to these buildings from > the Maa-amet database (nothing that can't be imported again) > > 2. Proceed with the Maa-amet building import as usual > > > > It will solve: > > 1. The license issue (if there is any) > > 2. The quality issue (if you agree there is an issue) > > 3. Will update the map in general, for example the demolished buildings will > be removed from OSM. > > > > For buildings which geometry was changed by other contributors after the > initial import - we can assume both license and quality issues were solved > since they no longer contain the imported geometry. I know it's a grey field, > and I'm not sure it works like that, but at least these buildings do have some > excuse to stay. > > > > For buildings which geometry was NOT changed, but some POI tags were added - > let them stay for now and discuss it later if needed. I suspect it will be a > rare case, but the exact number is unknown right now. > > > > Questions: > > 1. Has anyone else digged into the issue, maybe asked verbatium himself? > > 2. Can anyone confirm that the import indeed has the license problem? > > 3. Is the proposed plan good? (in case if you agree that it needs to be > fixed) > > > > -- > > SviMik > > ___ > > Talk-ee mailing list > > Talk-ee@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk-ee > > -- Svjatoslav Mikhailov ___ Talk-ee mailing list Talk-ee@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ee
