I can run all the shapefiles through qgis simplify tool if this resolves the issue...
On Fri., Jan. 18, 2019, 4:08 p.m. Nate Wessel <[email protected] wrote: > With default settings in JOSM, sure. In the import I was working on, we > used a Douglas-Peucker algorithm with a 20cm threshold (before the import > started) and it worked beautifully. We had many points that seemed to have > been introduced in the shapefiles as some kind of data artifact - they > didn't add any detail to the shape at all. This procedure removed almost > all of them with no discernible reduction in quality. > Nate Wessel > Jack of all trades, Master of Geography, PhD candidate in Urban Planning > NateWessel.com <http://natewessel.com> > > On 1/18/19 4:03 PM, James wrote: > > dare you to run simplify tool on anything remotely round, it will make it > look like garbage > > On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan <[email protected] > wrote: > >> The import mailing list was pointed to the correct page of the wiki. The >> initial post was to say this is what we were thinking of and there was a >> comment saying we needed to change the comment line. >> >> >There is no mention of this proposed import on the import catalogue >> >> >> The import process was reviewed by the person who set up the Ottawa >> import did we miss that step on the Ottawa import as well? Neither was it >> raised as a concern on the import mailing list. I think this is very minor >> and can be corrected. >> >> We learnt a fair bit on the Ottawa import and my expectation is since we >> are using experienced mappers to do the import conflation would be either >> handled by them or the building not imported. We aren't using new mappers >> in a mapathon here and with experienced mappers then I think you have to >> trust them. The world isn't perfect. Think in terms of service level. >> >> >There are 2X more nodes than needed to represent the building accurately. >> >> The problem with correcting this is you are introducing approximations. >> This will vary according to the source and this can be simplified or >> corrected once its in OSM. I think this is a different issue of a >> mechanical edit that needs to be considered separately. >> >> If we are concerned with database size then I suggest we change the >> instructions to say put the source comment on the change set rather than on >> the building outline. >> >> Cheerio John >> >> >> Nate Wessel wrote on 2019-01-18 3:06 PM: >> >> John, >> >> You seem to be playing the long game with this data - it sounds like >> you've been working with this a lot longer than I have, and you've put in >> the time and effort to help make this actually-quite-incredible dataset >> available to us. I don't want to stop the import from happening - quite the >> opposite. I just want to make sure that the time is taken to do this right. >> OSM deserves that. Your (our) long awaited victory will be the sweeter for >> our patience now. >> >> There are several specific issues I see where the I's are not crossed, >> nor the t's dotted. I've mentioned several already, so I'll try to be brief >> (I really need to get back to working on my dissertation). >> >> 1) There was extremely limited discussion on the imports mailing list. >> The initial email did not make clear the scope of the project. I read the >> email and did not think twice at it, thinking it was entirely about Ottawa. >> The link in that email was actually to the Ottawa import, and not this one, >> which seems to have been only in draft at the time. >> https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html >> As such, this project has NOT been reviewed by the imports list, which is >> a requirement for proceeding with the import. >> >> 2) There is no mention of this proposed import on the import catalogue ( >> https://wiki.openstreetmap.org/wiki/Import/Catalogue) >> which is required in the imports guidelines. I suspect many other >> guidelines have not been followed. >> >> 3) The wiki page describing the import is not adequate to assess the >> quality of the data or of the proposed import. See for example: >> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks >> The import guidelines call for a description of how conflation will be >> handled. The fact that two of the major importers seem to have a >> substantial disagreement about how to handle existing data indicates this >> was not well discussed and I can see that it isn't well documented. >> >> 4) The buildings need to be simplified, quite a bit actually. Most >> buildings have multiple nodes representing straight lines. This bloats the >> database and makes things harder to edit by hand later. There are probably >> 2x more nodes than are needed to represent the data accurately, making it >> harder for editors and data consumers to work with down the road.This is a >> simple fix that will save countless hours later. >> >> ... I could go on, but I think this is plenty sufficient to justify >> pressing pause on all this. >> >> Again, I don't in any way want to disrespect the work that has gone into >> this effort already. We're all volunteers here and I know how much time >> this all takes. However. importing all/most of the buildings in Canada is a >> monstrously large task, which will have to dance around a lot of people's >> toes. We should expect this to take a really damn long time if we're going >> to do it right. We need to have the patience to learn from experience, from >> critique, and from the wisdom of the people who've learned from flawed >> imports in the past and have devised guidelines and processes so that we >> can have better experiences with this in the future. >> Nate Wessel >> Jack of all trades, Master of Geography, PhD candidate in Urban Planning >> NateWessel.com <http://natewessel.com> >> >> On 1/18/19 2:24 PM, john whelan wrote: >> >> My background is I'm a retired civil servant who has written and overseen >> procurement documents and fairly large procurements. Dotting the is and >> crossing the Ts are my speciality. >> >> There are two parts to an import. The first part is the part played by >> the import mailing group. They confine themselves to is the license >> correct and do you have a reasonable plan. In this case the license is one >> of the few that has been confirmed by the Legal Working Group of >> OpenStreetMap and as such no questions were raised about it on the import >> mailing list. We have methodology that has been used before successfully >> with the Ottawa building outline import. There were major discussions both >> on talk-ca and the import mailing group before that import took place and >> we took note of the issues raised and addressed them. The licensing issue >> goes back about eight years to when I was talking to Federal Government >> Treasury Board and explaining their Open Data license did not align with >> OSM. That is why their license is now known as 2.0. >> >> The second part is the local group makes the decision to import they are >> the authority no one else. >> >> Apparently you were not part of the talk-ca when the discussions took >> place which would have been the time and place to raise concerns. >> >> When the Ottawa import was done there were one or two places where the >> existing buildings and the import overlapped. In the instructions on the >> import there are instructions to cover this. Specifically there is a >> validation step. I seem to recall the error rate was of the order of 1% >> and I expect this latest batch to be roughly the same. >> >> If you can identify which municipalities data is of poor quality then I'm >> sure we can remove these. For the most part these are from the foundation >> plans recorded by the municipality using professional surveying techniques. >> >> Would you like to clarify exactly where I failed to dot the Is and cross >> the Ts please. >> >> Many Thanks >> >> John >> >> >> >> On Fri, 18 Jan 2019 at 13:37, Nate Wessel <[email protected]> wrote: >> >>> Hi John, >>> >>> As Steve has said, you seem to be the only one suggesting that thousands >>> of import committees might need to be formed. Certainly I'm not suggesting >>> that. >>> >>> My understanding of OSM import procedure (and wiki-style projects more >>> generally) is that imports should operate in an essentially consensual way >>> where possible. The goal is to build consent and bring people on board with >>> a project or a change by addressing their concerns in a meaningful and >>> respectful way. >>> >>> I think that I have made some substantive and troubling claims about the >>> quality of the data being imported. I've pointed out that this project has >>> not followed the import procedures that were produced by a community of >>> mappers larger than just those in Canada. >>> >>> So to respond to your implication, I am in some sense the one reviewing >>> the project, just as I would welcome you to find ways that my own >>> contributions could be better. If you want my credentials for reviewing >>> your work, here they are: >>> >>> 1) I am an active contributor to OSM in Toronto, where I live (and >>> elsewhere) >>> >>> 2) I am currently helping to lead a building import in Hamilton County >>> Ohio that has better addressed some of the issues I see this import >>> struggling with. I can help you do the same. >>> >>> 3) I've been doing research in GIS for a long time now, though I don't >>> need that to tell you that the issues I've described are hardly >>> insurmountable technically or even all that difficult to fix. It would take >>> maybe one day's hard work to get the technical side of this right. >>> >>> I think Canadian OSMers will agree that we can take a pause to get >>> things right on such a massive import. If they don't - if I'm shouted down >>> or better, if my critiques are adequately addressed, then I will leave you >>> to finish the project in peace. I might even lend a hand if all goes well, >>> as I sincerely hope it does :-) >>> >>> Best, >>> Nate Wessel >>> Jack of all trades, Master of Geography, PhD candidate in Urban Planning >>> NateWessel.com <http://natewessel.com> >>> >>> On 1/18/19 1:11 PM, john whelan wrote: >>> >>> I know of no other way to contact him but he made an interesting comment >>> that the project is on hold in the wiki pending review. >>> >>> Would he care to comment on who is supposed to be reviewing the project? >>> >>> My understanding is that the import was raised in talk-ca before it >>> commenced for comment and these were generally favourable. I took that as >>> the local mappers to Canada had been consulted and they are the "local >>> mappers" authority in this case. >>> >>> I understand he has concerns about local mappers making decisions but in >>> Canada we have been importing similar data through CANVEC for some time. >>> CANVEC data comes from a number of sources including municipal data. >>> >>> Is he suggesting that each of the 3,700 municipalities in Canada should >>> form a group of local mappers who can make individual decisions on whether >>> their municipal data should be imported and we should end up with 3,700 >>> import plans? >>> >>> Thanks John >>> >>> >>> >>> _______________________________________________ >>> Talk-ca mailing >>> [email protected]https://lists.openstreetmap.org/listinfo/talk-ca >>> >>> _______________________________________________ >>> Talk-ca mailing list >>> [email protected] >>> https://lists.openstreetmap.org/listinfo/talk-ca >>> >> >> -- >> Sent from Postbox >> <https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach> >> _______________________________________________ >> Talk-ca mailing list >> [email protected] >> https://lists.openstreetmap.org/listinfo/talk-ca >> >
_______________________________________________ Talk-ca mailing list [email protected] https://lists.openstreetmap.org/listinfo/talk-ca
