[Talk-us] shawnee county landuse
Hi, Paul Norman pointed out that I forgot to mail people about my work on topeka. I have recently moved here and have been working on fixing things up. I am working on the topekabikelanes.org as a volunteer and have put in all the bikelanes as proposed routes. I am using the county gis system http://gis.snco.us/publicgis/ps/ and have written a ruby script to pull out the house numbers and land useage data. This data is being check street by street by me personally and I am repairing bad names and tiger garbage. About a separate account, I dont know how I am going to easily manage that, anyone have an idea to do that in josm? In any case i am not mass importing, but doing street by street what I would be doing manually. please advise if there is anything else you need to know. code is here : https://github.com/h4ck3rm1k3/shawnee-county-osm-landuse thanks, mike -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org Saving wikipedia(tm) articles from deletion http://SpeedyDeletion.wikia.com Contributor FOSM, the CC-BY-SA map of the world http://fosm.org Mozilla Rep https://reps.mozilla.org/u/h4ck3rm1k3 Free Software Foundation Europe Fellow http://fsfe.org/support/?h4ck3rm1k3 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
If anyone has a problem relating to this I don't mind fixing it. The way to do it is simply to download all the data within the bounding box, then use either the search or the authors dialogue to select all the data uploaded by that particular user and remove. Then the data can be re-uploaded :) Cheers, ingalls On Sat, Dec 15, 2012 at 1:18 AM, Paul Norman penor...@mac.com wrote: I notice that several of you are using uploads that involve 50k nodes per changeset. ** ** What is your revert plan if one of these long extended uploads gets interrupted? The JOSM reverter plugin will not handle this correctly. ** ** *From:* Jason Remillard [mailto:remillard.ja...@gmail.com] *Sent:* Thursday, December 13, 2012 8:12 PM *To:* talk-us@openstreetmap.org; impo...@openstreetmap.org *Subject:* [Talk-us] MassGIS Building Import - process ** ** Hi, I update the wiki http://wiki.openstreetmap.org/wiki/MassGIS_Buildings_Import It includes all of the emails from tonight. - The import account will be restricted to directly uploading the data. Any post fixes should be done with the normal account. - There will be a new upload tomorrow with simplified data, and the full massgis data osm replaced with an overlapping osm. This will be saved away for later incase anybody wants to do merges after this project is done. I will do Groton for sure Only two remaining issue - If we should restrict manual importing to a small group of people. I have gotten several feedback messages that letting everybody do it has *** * turned out not to be a great idea on other imports. We are up to 5 people now, all of them have zillion changes to the map. This is not a problem yet. Honestly, I am not sure if people will be beating down the doors to help or not... - How hard do we work on the manual imports. Do we stop doing them if the data looks really good, and just run the script, or do we keep doing the manual imports until we tire of it. Running the script sooner, gets the map better quicker, and lighter OSM users can probably handle cleaning up the small 1/500 errors. After this, I think we are ready to roll this up and get to work. Thanks Jason. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
Hi, On 14.12.2012 05:12, Jason Remillard wrote: - How hard do we work on the manual imports. Do we stop doing them if the data looks really good, and just run the script, or do we keep doing the manual imports until we tire of it. Running the script sooner, gets the map better quicker, and lighter OSM users can probably handle cleaning up the small 1/500 errors. My usual lament: 1. The map is not better just because more is on it. 2. If you cannot be bothered to fix problems then why should others be? What is your plan for growing the community to a point where it can maintain the data you plan to dump onto OSM? It is a common scenario to see importers aim for a quick win (look how much better we've made the map) and then let others clean up the mess. In programming, this is called technical debt - in order to be able to present nice results quickly you add some problems to your code which have to be cleaned out later. Sometimes there's a good reason for incurring such debt (e.g. because a deadline forces you to presents results quickly no matter how rotten your code is on the inside). What is your external deadline that would prevent you from doing things right? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
Paul - I've added a few comments and questions about changeset size and revert policies on the Import Guidelines Plan Outline wiki pages. Are there any recommended changeset size limits and/or revert plan practices? - Jeff On Sat, Dec 15, 2012 at 6:43 AM, nicholas ingalls nicholas.inga...@gmail.com wrote: If anyone has a problem relating to this I don't mind fixing it. The way to do it is simply to download all the data within the bounding box, then use either the search or the authors dialogue to select all the data uploaded by that particular user and remove. Then the data can be re-uploaded :) Cheers, ingalls On Sat, Dec 15, 2012 at 1:18 AM, Paul Norman penor...@mac.com wrote: I notice that several of you are using uploads that involve 50k nodes per changeset. ** ** What is your revert plan if one of these long extended uploads gets interrupted? The JOSM reverter plugin will not handle this correctly. ** ** *From:* Jason Remillard [mailto:remillard.ja...@gmail.com] *Sent:* Thursday, December 13, 2012 8:12 PM *To:* talk-us@openstreetmap.org; impo...@openstreetmap.org *Subject:* [Talk-us] MassGIS Building Import - process ** ** Hi, I update the wiki http://wiki.openstreetmap.org/wiki/MassGIS_Buildings_Import It includes all of the emails from tonight. - The import account will be restricted to directly uploading the data. Any post fixes should be done with the normal account. - There will be a new upload tomorrow with simplified data, and the full massgis data osm replaced with an overlapping osm. This will be saved away for later incase anybody wants to do merges after this project is done. I will do Groton for sure Only two remaining issue - If we should restrict manual importing to a small group of people. I have gotten several feedback messages that letting everybody do it has ** ** turned out not to be a great idea on other imports. We are up to 5 people now, all of them have zillion changes to the map. This is not a problem yet. Honestly, I am not sure if people will be beating down the doors to help or not... - How hard do we work on the manual imports. Do we stop doing them if the data looks really good, and just run the script, or do we keep doing the manual imports until we tire of it. Running the script sooner, gets the map better quicker, and lighter OSM users can probably handle cleaning up the small 1/500 errors. After this, I think we are ready to roll this up and get to work. Thanks Jason. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Jeff Meyer Global World History Atlas www.gwhat.org j...@gwhat.org 206-676-2347 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
On Sat, Dec 15, 2012 at 7:50 AM, Frederik Ramm frede...@remote.org wrote: 2. If you cannot be bothered to fix problems then why should others be? What is your plan for growing the community to a point where it can maintain the data you plan to dump onto OSM? What's an acceptable error rate? 0? What's the error rate for manual entry? I'd suggest that if Jason hits 1 error in 500 whatevers, that that's lower than manual entry. What are some specific examples of costs expected to be incurred by importing the building outlines the Massachusetts OSM community has identified? Could concerns about errors be addressed by a manual, building-by-building QA process that might address any error issues? What can he *do* to address your concerns? Rather than addressing the imports=good vs. imports=bad discussion, what are the steps Jason hasn't addressed that should be included in order to make sure that we are minimizing the potential for introducing problems into the OSM database. - Jeff ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] Fwd: osm and K.S.A. 45-220(c)(2)
the only issue that we might have with the data imported is this law, but trulia and zillow and all those property databases also have all this data as well. Also I found this, a nice set of contours and orthophotos http://www.topeka.org/maps/gis_map.htm mike -- Forwarded message -- From: Mike Dupont jamesmikedup...@googlemail.com Date: Sat, Dec 15, 2012 at 2:02 PM Subject: osm and K.S.A. 45-220(c)(2) To: Licensing and other legal discussions. legal-t...@openstreetmap.org Hello, Would putting the data in osm be in violation of the kansas law K.S.A. 45-220(c)(2)? http://kansasstatutes.lesterama.org/Chapter_45/Article_2/45-230.html K.S.A. 45-220(c)(2): the requester does not intend to, and will not: (A) Use any list of names or addresses contained in or derived from the records or information for the purpose of selling or offering for sale any property or service to any person listed or to any person who resides at any address listed; or (B) sell, give or otherwise make available to any person any list of names or addresses contained in or derived from the records or information for the purpose of allowing that person to sell or offer for sale any property or service to any person listed or to any person who resides at any address listed. thanks mike -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org Saving wikipedia(tm) articles from deletion http://SpeedyDeletion.wikia.com Contributor FOSM, the CC-BY-SA map of the world http://fosm.org Mozilla Rep https://reps.mozilla.org/u/h4ck3rm1k3 Free Software Foundation Europe Fellow http://fsfe.org/support/?h4ck3rm1k3 -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org Saving wikipedia(tm) articles from deletion http://SpeedyDeletion.wikia.com Contributor FOSM, the CC-BY-SA map of the world http://fosm.org Mozilla Rep https://reps.mozilla.org/u/h4ck3rm1k3 Free Software Foundation Europe Fellow http://fsfe.org/support/?h4ck3rm1k3 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
On 15 December 2012 20:09, Jeff Meyer j...@gwhat.org wrote: Paul - I've added a few comments and questions about changeset size and revert policies on the Import Guidelines Plan Outline wiki pages. Are there any recommended changeset size limits and/or revert plan practices? One good practice is not to revert data that is not known to be wrong. If a big changeset fails halfway through it's possible to fix the remaining part to use the nodes that have been uploaded and continue, rather than delete the 1000s of nodes just to create new ones in the same places. You can probably now do that in JOSM by downloading the changeset containing the orphaned nodes, opening in JOSM together with the data being uploaded and telling the validator to fix all duplicate nodes. Myself I've been using the python scripts at http://wiki.openstreetmap.org/wiki/Upload.py in such situations, although the api is much more stable now than it was a couple years ago. The other good practice, but possibly not usable with JOSM alone, is not to let the program upload the naked nodes in bulk and then the buildings in bulk. You can sort the elements in such a way that every 50k element changeset contains say 45k nodes and 5k ways. The scripts let you limit the number of elements in a chunk which the next chunk depends on to the minimum (optimally 0), this way there's no risk of a passer by spotting orphan nodes and deleting some causing you conflicts in your next chunk. Cheers ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] shawnee county landuse
1. You need to have a wiki page and list the data source on the import catalogue 2. What is the license that this data is under? I did not see a copyright statement. 3. What tagging are you proposing? 4. The last few times someone has proposed importing property lot data the consensus has been that that type of data shouldn't be imported into OSM. Why is this different? 5. The consultation should have been done before the import. What do you propose doing with the data you've already imported? I've cc'ed imports@ because they also need to be part of the consultation. From: Mike Dupont [mailto:jamesmikedup...@googlemail.com] Sent: Saturday, December 15, 2012 6:05 AM To: talk-us@openstreetmap.org; Paul Norman Subject: [Talk-us] shawnee county landuse Hi, Paul Norman pointed out that I forgot to mail people about my work on topeka. I have recently moved here and have been working on fixing things up. I am working on the topekabikelanes.org as a volunteer and have put in all the bikelanes as proposed routes. I am using the county gis system http://gis.snco.us/publicgis/ps/ and have written a ruby script to pull out the house numbers and land useage data. This data is being check street by street by me personally and I am repairing bad names and tiger garbage. About a separate account, I dont know how I am going to easily manage that, anyone have an idea to do that in josm? In any case i am not mass importing, but doing street by street what I would be doing manually. please advise if there is anything else you need to know. code is here : https://github.com/h4ck3rm1k3/shawnee-county-osm-landuse thanks, mike -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org Saving wikipedia(tm) articles from deletion http://SpeedyDeletion.wikia.com Contributor FOSM, the CC-BY-SA map of the world http://fosm.org Mozilla Rep https://reps.mozilla.org/u/h4ck3rm1k3 Free Software Foundation Europe Fellow http://fsfe.org/support/?h4ck3rm1k3 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
On 15 December 2012 22:45, Paul Norman penor...@mac.com wrote: From: andrzej zaborowski [mailto:balr...@gmail.com] Subject: Re: [Talk-us] MassGIS Building Import - process On 15 December 2012 20:09, Jeff Meyer j...@gwhat.org wrote: Paul - I've added a few comments and questions about changeset size and revert policies on the Import Guidelines Plan Outline wiki pages. Are there any recommended changeset size limits and/or revert plan practices? One good practice is not to revert data that is not known to be wrong. If a big changeset fails halfway through it's possible to fix the remaining part to use the nodes that have been uploaded and continue, rather than delete the 1000s of nodes just to create new ones in the same places. You can probably now do that in JOSM by downloading the changeset containing the orphaned nodes, opening in JOSM together with the data being uploaded and telling the validator to fix all duplicate nodes. This hasn't come up for me on an import but I've tried it with normal mapping. I don't believe you can do a fix all on duplicate nodes and instead have to resolve them all individually With a current JOSM it seems you can select all the errors and click fix, or you can select the Other duplicate nodes category and click fix, I just checked. But JOSM will add each pair of merged nodes as an individual operation in undo history. I noticed it also merges the dupe nodes when you're merging two layers rather than copy from one layer and paste onto another. Myself I've been using the python scripts at http://wiki.openstreetmap.org/wiki/Upload.py in such situations, although the api is much more stable now than it was a couple years ago. What's your workflow? Do changes in JOSM, save and then pass to the scripts? Yes. The osm2change script understands the JOSM format and produces and .osc. With the TIGER name expansion the bot produced .osc files directly which were reviewed in a text editor. The other good practice, but possibly not usable with JOSM alone, is not to let the program upload the naked nodes in bulk and then the buildings in bulk. You can sort the elements in such a way that every 50k element changeset contains say 45k nodes and 5k ways. The scripts let you limit the number of elements in a chunk which the next chunk depends on to the minimum (optimally 0), this way there's no risk of a passer by spotting orphan nodes and deleting some causing you conflicts in your next chunk. The best way to do this in JOSM alone is to only merge in 1-5k nodes+ways at a time, review them, then upload. This also avoids most of the problems above. I would only ever do a 50k object changeset in very limited circumstances where I am confident that it is safe to do so. Even then I'd try to keep it under 25k normally. For normal imports I'd suggest 10k as a soft limit. Yes, good points. Cheers ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
Jeff Meyer j...@gwhat.org writes: On Sat, Dec 15, 2012 at 7:50 AM, Frederik Ramm frede...@remote.org wrote: 2. If you cannot be bothered to fix problems then why should others be? What is your plan for growing the community to a point where it can maintain the data you plan to dump onto OSM? What's an acceptable error rate? 0? What's the error rate for manual entry? I'd suggest that if Jason hits 1 error in 500 whatevers, that that's lower than manual entry. Well put. Before the import, a vast number of buildings that actually exist were not shown on the map. To me, that is a greater problem than a tiny number of non-buildings rendered as buildings. One can argue about the weight of actual buildings not shown vs non-buildings shown in computing a total map quality function, but by all accounts of people who have looked at the data the quality is very high. And the non-buildings shown are things are typically thing that are structure-like but not really structures, which is exactly the kind of thing that I've added by hand as builings when maybe I shouldn't. The community in Mass is actually growing, and I haven't heard from anyone local who is opposed to what's happening. This process is causing us to know each other better. So I think random complaining at Jason for you are importing so you must be harming the community is uncalled for, unsubstantiated, and harmful to building community. In my view, community is about whether people want to belong and want to work with each other, and I see the current situation as a net plus. pgpSKUvwmGOiS.pgp Description: PGP signature ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
Jeff Meyer j...@gwhat.org writes: Paul - I've added a few comments and questions about changeset size and revert policies on the Import Guidelines Plan Outline wiki pages. Are there any recommended changeset size limits and/or revert plan practices? Tools support for reverting is a fair point. But, I'd argue that it's a bug in JOSM if one can upload a single changeset that the reverter can't undo, and if so (I may be confused) this should be fixed in JOSM rather than be a special manual requirement. pgpphR8cNggpV.pgp Description: PGP signature ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MassGIS Building Import - process
From: Greg Troxel [mailto:g...@ir.bbn.com] Subject: Re: [Talk-us] MassGIS Building Import - process Jeff Meyer j...@gwhat.org writes: Paul - I've added a few comments and questions about changeset size and revert policies on the Import Guidelines Plan Outline wiki pages. Are there any recommended changeset size limits and/or revert plan practices? Tools support for reverting is a fair point. But, I'd argue that it's a bug in JOSM if one can upload a single changeset that the reverter can't undo, and if so (I may be confused) this should be fixed in JOSM rather than be a special manual requirement. It's not a bug in JOSM. JOSM gives you lots of way to shoot yourself in the foot, this is due to an interaction between large changesets, the rails port, database/serialization times on the changeset /download call and imports. Because it's pretty much import-specific it's not too serious an issue if people are following the guidelines because they require you to have the knowledge of how to fix your mistakes. The checklist suggests developing a revert plan and the guidelines specifically state If you don't know how to revert an import, don't do the import in the first place. All of this being said, an item on my to-do list is to propose reducing the maximum changeset size. Right now it is too easy to make a changeset that only specialized ill-documented tricky to use tools can download. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
