[Talk-us] Separate sidewalk vs crossing in the middle of 45 mph road
Hello, As per https://wiki.openstreetmap.org/wiki/Sidewalks#Sidewalk_as_separa te_way I started mapping the sidewalks that run along the main roads, but are physically separated from the carriage with more than a curb - http://overpass-turbo.eu/s/gKo Should the highway=* with separate highway=footway/footway=sidewalk be tagged as sidewalk=no? Now, routing via Grasshopper and Mapzen will direct you to the road without a sidewalk just because you want to turn before the crossing (instead of going a safer route via pedestrian crossing with traffic signals): https://graphhopper.com/maps/?point=42.506496%2C-83.42391=42.4836 81%2C-83.417076=en- US=foot=fastest=true=Omniscale http://www.openstreetmap.org/directions?engine=mapzen_foot=42.505 9%2C-83.4242%3B42.4837%2C-83.4172#map=17/42.49893/-83.41971 The sidewalk "islands" are in the real world as well, forcing you to seek alternative route through the grass/bushes or cross a 45 mph street. signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi, As Massachusetts slowly migrates back into winter, here is the status update on the address import for Boston, MA. License: As it turns out, all public City of Boston data is licensed CC BY 4.0, which requires attribution. I have sent additional clarification questions on whether the 'Contributors' page link/import changeset source tag is sufficient to City of Boston officials as per the wiki instructions. Neighborhoods: I have completely split: Back Bay, Bay Village, Beacon Hill, Chinatown, Dorchester, Financial District, Government Center, Hyde Park, Jamaica Plain, South Boston, South End, West End, and Roxbury. As of yesterday's merge, we have 75687 uniquely identified buildings (up from 64K 3 weeks ago) and 1449 more are still waiting to be split (2%). SAM contains address information on 89065 distinct objects with addresses (some of them are buildings, some of them are parks or references to buildings that were demolished a long ago), and we have 85% overlap of OSM buildings and SAM objects. Yay! -- https://wiki.openstreetmap.org/wiki/Import/Boston_Street_Address_Manage ment_%28SAM%29_Import signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi all, Just an update on the current status. 1. I am waiting for a response from Boston GIS regarding the license terms (MassGIS != Boston GIS, as I recently realized). 2. At this point the code will only be minimally tweaked as I have implemented everything I originally planned: - Standalone buildings w/o numbers are tagged - Standalone buildings w/ numbers differing from SAM data are flagged for further manual updates - Terraced buildings are marked with gpx waypoints/.osn note entries for manual splitting/surveying. - source:addr=survey buildings are skipped. - Buildings containing an entrance that has an addr:housenumber are skipped (most likely a manual survey). - Address nodes are added if there are multiple addresses for a building within one tax parcel. Address nodes are in a separate .osc file; if import is approved, these will be the last thing to be imported. 3. I am going through the neighborhoods splitting the buildings. If anybody is aware of an algorithm that can make this process easier (buildings may span multiple parcels, and sometimes spill over parcels they have no business being into) I'll be glad to hear about it. I talked to Lars Ahlzen @ OSM Meetup this week and confirmed that adding addr:city and addr:postcode to a building/address node is a good thing. -- https://wiki.openstreetmap.org/wiki/Import/Boston_Street_Address_Manage ment_%28SAM%29_Import signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi Jason, all. I added the addr:city to the tags to use w/o confirming first - what is the balance between adding the address information directly on the building as opposed to using the boundaries? I suppose that for the ease of processing the building will need to have as much information as possible, but then we will have two sources of truth e.g. for city or zipcode - boundaries and the node. Now, current status: I have just terraced Back Bay (a historical district in Boston, old narrow houses, around 1000 of them) and found this to be less fun than I imagined :) Additionally a look at South Boston shows that there are less building ranges, and more building numbers that point to the same building (e.g. number 45 is on first floor, 47 is on the second). As much as I'd hate to do that, there appears to be no other way to handle this than adding the address node, as I saw done in NY and Seattle (and how e.g. Here maps handles it - no buildings, just numbers on the ground). Now, that also means that I need to start operating on the tax parcel shapefile to verify whether the building needs to be split or an address node needs to be added. I added an exception for buildings with source:addr=survey, as I found that it is of no use trying to repeatedly mark a building which was manually tagged and verified to have a different number than the official one as "fixme". So far there are ~3 buildings with this tag, but there will be more as I am going through the dataset and buildings on the ground. -- https://wiki.openstreetmap.org/wiki/Import/Boston_Street_Address_Manage ment_%28SAM%29_Import signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi Jason, On Mon, 2016-03-21 at 20:42 -0400, Jason Remillard wrote: > Hi Roman, > > The city of Boston building data set for buildings has address. > > http://bostonopendata.boston.opendata.arcgis.com/datasets/492746f09dd > e475285b01ae7fc95950e_1 Interesting, so that's where the tax parcel basemap buildings are coming from... > It seems like they have already figured out what address goes on what > building. Should this data set be used rather than the parcel data > set? We are not using the parcel data set, instead, we are using a service build specifically to point to street addresses - http://bostonopendata .boston.opendata.arcgis.com/datasets/b6bffcace320448d96bb84eabb8a075f_0 (Live Street Address Management (SAM) Addresses), which is, well, live and appears to have changed every time I fetched the geojson feed. > The licensing link says the following, it is kind of weird. > > "The City of Boston recognizes the value and benefit gained by > sharing GIS data. ... The GIS data is updated > and modified on a regular basis and users are encouraged to report > any errors to the City." At some point I found a notice that since the data was paid for by the taxpayers, it is freely available to the public. I hope I'll meet someone from MassGIS this week to clarify the statement above. > I suggest that you move forward with the building splitting, since it > is a manual process, it can proceed like a normal mapping activity. Yup, that's what I am doing now. > While the buildings are being cut up, we can work on address import > and make sure it is good shape. Before I would be comfortable with > the import, I would like to see some sample OSM files we can load > into JOSM to look over. I have been regenerating the .osc files daily (new batch in an hour) when new data is available or the code is optimized. All the files are linked to from the wiki page - https://wiki.openstreetmap.org/wiki/Impo rt/Boston_Street_Address_Management_%28SAM%29_Import#OSM_Data_Files. I'd like to point out that only -unique.osc are considered for import, the -fixmes.osc are for further investigation and additional processing. As per the "Buildings" dataset you mentioned above, that does look like a good source, but I did a spot-check on various buildings I've surveyed: * Object ID: 91094 - no house number (should be 32-36 John A Andrew Street) * Object 90372 - 110 McBride (should be 108-110) - and generally building ranges are missing, so there are e.g. 5 "378 Riverway"s since they are located on the same parcel. The geometry information, however, looks very yummy. It's not directly usable since the various building parts are provided as separate objects probably linked to by the BUILDING_I. Now, if we at some point decide to replace the Boston geometry information, that would be a great dataset. But I feel this is far more involved than adding the house numbers for basic navigation needs which is what I am after. And even if we do, there are newer shapefiles based on Orthoimagery/LIDAR which has the newer building shapes, and geometry is completely different, buildings are not split and have no addresses. -- Sincerely, Roman signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi, all! Going back to the point... I stopped linking to the .osn files in order to minimize the confusion. Instead, every neighborhood gets their .gpx file with the details of the issues encountered at that particular address: - Building is missing. - Building has more than one address. - Building has a street name that is not known to OSM or has not been added to the mapping. This will allow locating the issues quicker while surveying. I have also generated the .pbf file containing the last export of US, MA from Geofabrik + unique house numbers. Since I am using OsmAnd, I have also generated the .obf map. Basically, that's how OSM would look like if the current list of changes is applied. https://wiki.openstreetmap.org/wiki/Import/Boston_Street_Address_Manage ment_%28SAM%29_Import#OSM_Data_Files There was a question of the data accuracy earlier - I've spent the last 3 days spot-checking the building numbers and I have to say they are pretty accurate. Some buildings, however, have picked one number from the range assigned, and this will have to be rectified manually as well. The data from Boston Tax Parcel may help with that, but it's license is not completely clear. I found a couple of issues while generating the datasets/surveying the locations: * Some of the addresses from MassGIS point to parks, monuments, or buildings are located a few feet away. While it may possible to assign the marker to the building nearby, I'd prefer not to do the guess work and leave it for refining in the future. * Original OSM street names must have been imported from MassGIS roads, and a few of these don't correspond to the signs (something as minor as Conry Crescent Street vs Conry Crescent or slightly more interesting Glenvale Terrace vs Chestnut Terrace). So there are clusters of buildings in .gpx file pointing to possible street naming issues, which will have to be manually correlated with the signs on the streets. * Multipolygon buildings are not supported by my script, so there will be up to 48 'missing building' false positives for the whole Boston. Overall, I feel pretty good about the quality of the data and would like to hear more about issues that I may have missed. -- Sincerely, Roman Yepishev signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi Greg, all. I'd like to provide some information on the import I did not share initially to make my intentions clear (so far I may be seen as dumping the data into OSM and running away). TL;DR = * I believe in an iterative approach to any project. * I want to make as many useful things as possible in the limited time I have. I want a strict deadline to mark the end of my active involvement and enable others to pick up without creating conflicts. * I want to improve the routing and location information for me and for others. * I have never wanted to introduce buildings with fixme in them into OSM, I will not import notes. * I will import data, fix buildings with issues, regenerate the data, rinse, repeat. * I will not overwrite the existing building numbers. * All the scripts I'm using are in public domain. Long version My background - First of all, I'm a development/operations guy with mostly web service background, and iterative approach to anything is basically the only way to do things there. Additionally I took part in a number of proprietary and open-source projects where the lack of completion deadlines forced the project to be ready when it is ready. Sometimes never. Motivation -- As I was walking around Boston using OsmAnd, I would often fail to locate the venues just because there were no building numbers on the street I'm supposed to go to. Now, I can still go along the street, but a number of times I got confused because the block no longer looked like it used to with the redevelopment going on. The only way is to bruteforce, expecting that the buildings will be in order. Building numbers "14 1/2" or "38R" are hard to locate unless you've been there already. As a novice driver I find lack of numbers and details also daunting, leaving me with no choice but to use Google Maps/Here maps, since I need to scan building numbers (sometimes located under a rock) as I am going. Rushing with a deadline --- Again, based on my involvement with various open-source projects, it is sometimes impossible to get anything vetted unless motivation is strong and a sense of urgency is introduced. I have to apologize for that here, as I should have explained it better in my first message. Importing artifacts --- My plan has always been to: 1. Import unique buildings. 2. Regenerate a set of .osc, .osn files, post them to wiki for review (if somebody wants to look at them) 3. Upload ONLY unique buildings into OSM (no fixmes, no notes, etc.). 4. Modify the rest of buildings so that they become unique (if possible, if not - skip them) 5. Add missing buildings, verify they are unique. 6. Repeat from 2. I will add this information to the wiki. Iterations -- The import will have to go through multiple iterations. At first I will import ONLY buildings that are already good to go and require no manual interaction. The quality of building polygons vary, and it may take much more time to reach 100% coverage (we may not even be able to reach it at all due to a changing nature of the cities), but marking 90% buildings will make locating the missing ones much easier. As I am trying to use OsmAnd more on my phone I find that presence of a lot of "height estimated" fixmes in Downtown Boston (you can enable OSM map helper) makes the map unreadable. Again, the -fixmes.osc files that you see in the wiki page are just for finding issues with existing buildings: - more than one addresses for a single building. - address is already provided, but differs from what the city expects it to be (e.g. 90 South St, Boston, MA - "Farnsworth House" has to be #100 according to two city sources, but the sign says #90, so it is #90 in OSM - https://www.openstreetmap.org/way/29939006 - I am planning to resurvey the 90-86 building above that.) I don't want to overwrite other people's hand-entered data, but I want to be able to see where the conflicts are. The notes must and will not be imported. They will be periodically regenerated so that it is possible to see how the OSM updates go (missing buildings added, existing buildings split). Tools - All the python/SQL scripts I am using to create the datasets are already available on bitbucket. You can perform the steps mentioned in README and you will end up with the same results as mine. I am yet to add the license text there, they are in a public domain. Please note the scripts originated from two all-nighters and I am refining them as I go. Closing statement - What I hope to accomplish is to make the routing work better for me, and as a great side effect, these changes will make OSRM pick the right buildings (or approximate locations), the search for the building in Nominatim will yield results instead of finding everything but the thing I searched for, and I am sure this will benefit the community as well. Before involvement with OSM I treated the routing
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Quick followup as I may have had the Tax Parcel 2015 dataset in mind. On Wed, 2016-03-16 at 08:57 -0400, Roman Yepishev wrote: > > > - The source data contains the building heights, you might want to > > import that in too. > Only number of floors is provided (which can be 2.5), not the height > itself. It is easy to add this though. My initial reading of https:// > wi > ki.openstreetmap.org/wiki/Key:building:levels suggested that > building:levels is not useful w/o height. There is no building height in http://bostonopendata.boston.opendata.ar cgis.com/datasets/b6bffcace320448d96bb84eabb8a075f_0, sorry about that. signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Boston, MA, USA addr:housenumber Import
The wiki now contains updated files that set the postcode and add a fixme tag to a building in case it already had the number that does not match the official information from SAM. https://wiki.openstreetmap.org/wiki/Import/Boston_Street_Address_Manage ment_%28SAM%29_Import#OSM_Data_Files On Tue, 2016-03-15 at 12:52 -0400, Roman Yepishev wrote: > (re-posting to both mailing lists since I failed to subscribe to > imports@ before sending) > > Hello OpenStreetMap folks, > > TL;DR: import wiki page: https://wiki.openstreetmap.org/wiki/Import/B > os > ton_Street_Address_Management_%28SAM%29_Import > > Boston, Massachusetts is relatively well mapped, however the > buildings > usually lack addr:housenumber and addr:street tags. > > Since the city is not built on a grid-based design, the streets can > wander around making finding the right house with OpenStreetMap a > challenge. City of Boston provides a sizable amount of data via Open > Government initiative. One of the things available freely is a Live > Street Address Management (SAM) dataset[1] that identifies the > location > of the buildings on the map. We can use that. > > As requested by Guidelines[2], I am writing to imports@ and talk-us@ > to > discuss the outlined import procedure for the data. I will be driving > the generation, upload, and validation. > > The wiki page contains the data on all the neighborhoods (sans 2 that > for some reason did not create a polygon - I'm investigating that), > so > you can load it locally and see what's going to happen. Please do not > upload these changes. > > My username is 'ryebread' on OSM, User:Rye on OSM Wiki. > > [1]: http://www.cityofboston.gov/open/ > [2]: https://wiki.openstreetmap.org/wiki/Import/Guidelines > > -- > Sincerely, > Roman Yepishev > ___ > Talk-us mailing list > Talk-us@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-us signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi Jason, Thanks for you message. Replies are inline. On Wed, 2016-03-16 at 08:02 -0400, Jason Remillard wrote: > Hi Roman , > > - The source addresses uses abbreviations (RD, ST,etc) are you > expanding them? There are only around 240 SAM streets that did not map to OSM streets exactly, these are now being manually added to the mapping table. > - The source addresses are capitalized, are you fixing that? Hm, source addresses are not capitalized, I am using the STREET_BODY + STREET_FULL_SUFFIX as a street name. "STREET_NUMBER":"72", "FULL_STREET_NAME":"Addington Rd", "STREET_ID":27,"STREET_PREFIX":" ", "STREET_BODY":"Addington", "STREET_SUFFIX_ABBR":"Rd", "STREET_FULL_SUFFIX":"Road", "STREET_NUMBER_SORT":72, "MAILING_NEIGHBORHOOD":"West Roxbury" "ZIP_CODE":"02132" > - How are you dealing with multiple addresses per building I am adding a local note highlighting that the the same way has more than one address to locate buildings that need to be split according to parcel map/retrace from Bing/MassGIS Orthoimagery. Such buildings are generated into separate -fixmes.osc files. > - How are you dealing with multiple buildings per address. Since only a single location is provided for a building (RELATION_TYPE=1) there is no multiple matching buildings per address. On a side note that may require additional survey on location in case a multipolygon is required. We can't deduce that from the source data. > - Unless you are working full time on this two weeks seems like not > enough time, it is 380,000 addresses. Please see the wiki page linked - https://wiki.openstreetmap.org/wiki/I mport/Boston_Street_Address_Management_(SAM)_Import there are about 5000 with issues out of 64K existing OSM buildings (~8%). At least until those two missing neighborhoods are found. Even if only unique buildings are imported it will yield a 92% increase of accuracy for building locations. The remaining time will be spent tweaking these 5000 buildings. And yes, I'm basically working on this full time. > - The source data contains the building heights, you might want to > import that in too. Only number of floors is provided (which can be 2.5), not the height itself. It is easy to add this though. My initial reading of https://wi ki.openstreetmap.org/wiki/Key:building:levels suggested that building:levels is not useful w/o height. > - I would like to see a sample osm/osc files. The .osc and .osn files can be found here - they are periodically regenerated as I am fixing the transformation scripts https://wiki.openstreetmap.org/wiki/Import/Boston_Street_Address_Manage ment_(SAM)_Import > > Thanks > Jason > > > > On Tue, Mar 15, 2016 at 12:52 PM, Roman Yepishev <r...@keypressure.com > > wrote: > > > > (re-posting to both mailing lists since I failed to subscribe to > > imports@ before sending) > > > > Hello OpenStreetMap folks, > > > > TL;DR: import wiki page: https://wiki.openstreetmap.org/wiki/Import > > /Bos > > ton_Street_Address_Management_%28SAM%29_Import > > > > Boston, Massachusetts is relatively well mapped, however the > > buildings > > usually lack addr:housenumber and addr:street tags. > > > > Since the city is not built on a grid-based design, the streets can > > wander around making finding the right house with OpenStreetMap a > > challenge. City of Boston provides a sizable amount of data via > > Open > > Government initiative. One of the things available freely is a Live > > Street Address Management (SAM) dataset[1] that identifies the > > location > > of the buildings on the map. We can use that. > > > > As requested by Guidelines[2], I am writing to imports@ and talk- > > us@ to > > discuss the outlined import procedure for the data. I will be > > driving > > the generation, upload, and validation. > > > > The wiki page contains the data on all the neighborhoods (sans 2 > > that > > for some reason did not create a polygon - I'm investigating that), > > so > > you can load it locally and see what's going to happen. Please do > > not > > upload these changes. > > > > My username is 'ryebread' on OSM, User:Rye on OSM Wiki. > > > > [1]: http://www.cityofboston.gov/open/ > > [2]: https://wiki.openstreetmap.org/wiki/Import/Guidelines > > > > -- > > Sincerely, > > Roman Yepishev > > > > ___ > > Imports mailing list > > impo...@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/imports > > signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Boston, MA, USA addr:housenumber Import
Hi Clifford, thank you for the message. The answers are inline below. On Tue, 2016-03-15 at 13:40 -0700, Clifford Snow wrote: > 1.) Automatically updating the street name to match the address > records is not advisable. My experience doing address imports, the > address information may not match the street signs. There may be two > different departments responsible for the information, accessor and > highway/street department. OK, that's a valid point. I will not overwrite the street info based on punctuation differences since then the MassDOT names will no longer match. > 2.) Is there addr:city and addr:postcode information available to add > with the import? Actually yes, there is the zip_code field in the source data. The city is Boston for all these buildings. I assumed that a ZIP code information is handled by some relation, and not attached to the building. Just read about addr:postcode on wiki and I don't see a reason why that can't be added to the buildings while we are at it. > > 3) Just looking at one neighborhood, Allston-Brighton, the .osc > contains building=yes. Does that mean if a building outline is tagged > building=residential or any other building tag it will be retagged > building=yes? The selection runs on anything that has a tag key building (and value is not 'no', just in case). No change of building type will be performed. > 4) Is it really necessary to upload that many notes during the > import? if a building is missing, can you add a node with the address > information instead of leaving a note? I should have been clear on that - the notes generated are not designed to be uploaded at all. It's just the only way I found that I can put a marker on the map and be able to see it in JOSM without creating a POI. I have updated the wiki page with the above clarifications. -- Sincerely, Roman signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] Boston, MA, USA addr:housenumber Import
(re-posting to both mailing lists since I failed to subscribe to imports@ before sending) Hello OpenStreetMap folks, TL;DR: import wiki page: https://wiki.openstreetmap.org/wiki/Import/Bos ton_Street_Address_Management_%28SAM%29_Import Boston, Massachusetts is relatively well mapped, however the buildings usually lack addr:housenumber and addr:street tags. Since the city is not built on a grid-based design, the streets can wander around making finding the right house with OpenStreetMap a challenge. City of Boston provides a sizable amount of data via Open Government initiative. One of the things available freely is a Live Street Address Management (SAM) dataset[1] that identifies the location of the buildings on the map. We can use that. As requested by Guidelines[2], I am writing to imports@ and talk-us@ to discuss the outlined import procedure for the data. I will be driving the generation, upload, and validation. The wiki page contains the data on all the neighborhoods (sans 2 that for some reason did not create a polygon - I'm investigating that), so you can load it locally and see what's going to happen. Please do not upload these changes. My username is 'ryebread' on OSM, User:Rye on OSM Wiki. [1]: http://www.cityofboston.gov/open/ [2]: https://wiki.openstreetmap.org/wiki/Import/Guidelines -- Sincerely, Roman Yepishev signature.asc Description: This is a digitally signed message part ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
