Re: [Talk-us] Address data for Miami Florida United States

[Leif Rasmussen]

Another update on the address data:
I managed to do the address data transformation by splitting the data up
into five files.  The data is now in ready to upload OSM format (except for
data errors).  All duplicates of existing addresses in the OSM database
have automatically been removed, leaving only missing addresses in the
dataset.  This means that the data could be uploaded to the OSM database
without creating duplicate addresses.
The data is available here:
https://drive.google.com/open?id=1DJGNdONqdTXMlA0e550ghsmpotqc4QM4

I split it up into five manageable files with roughly 100,000 points in
each.
The data itself has some minor issues.
1) Many "addr:city" tags are missing.  These can be added in manually
before upload by selecting all addresses with a certain postcode and adding
the city to them.  In the US, postcodes usually only have one city
associated with them, so adding the missing addr:city tags is much easier
because of this.
2) Some other tags are missing from about 50 addresses.  They don't have
"addr:street", "addr:postcode", or "addr:city".  Just "addr:housenumber"
and "addr:state".
Other than that, the data looks great!
I will fix up the wiki page and contact the imports mailing list sometime
this weekend if I can.

Levente Juhász,
Manually adding the addresses would take way too long.  Instead, a tasking
manager project should be created for organizing address upload.  If the
uploader wants, they can merge the data with existing features such as
buildings, but this is not required.  An upload without merging would
simply add all missing addresses around the existing ones.  I was thinking
that the tasks be about 5,000 - 10,000 addresses each.  This should allow
for quick and easy upload, even if Miami does not have very many
contributors interested in helping.
Thanks,
Leif Rasmussen
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] Address data for Miami Florida United States

[Levente Juhász]

Great stuff, Leif! I will catch up with this project over the weekend and
will provide some input.

I have access to a few workstations with a bunch of memory so I can help
out with converting/processing the input data if needed. Just let me know.
I don't think of the size of these datasets as an issue in terms of
processing (not even the buildings). As you said, splitting them up into
smaller chunks or using a totally different toolset would also work.

Finding enough people to review and put the data on the map will be a
bigger issue in Miami, imho. You guys are planning to implement an entirely
manual approach, correct?

Cheers,
Levente



On Fri, Sep 14, 2018, 4:06 PM Leif Rasmussen <354...@gmail.com> wrote:

> Update on the address data:
> I tried transforming the small data file (550,000 addresses) to OSM
> format, but my computer ran out of memory.  I will try using a more
> powerful computer later.  The transformation worked perfectly on a smaller
> file of 8000 addresses in Miami Beach, however.  I uploaded that sample
> file to a google drive folder
> .
> Simply download the file and drag and drop it into JOSM to view that
> addresses.  The address data source only had "addr:city" for some of the
> addresses, so that tag will have to be added manually based on postcode
> later.
> Thanks,
> Leif Rasmussen
>
> Also, the addresses with numbered street names (4th Street) have not been
> expanded (to eg. "Fourth Street").  The roads in OSM currently have the
> numbered versions (4th Street), so I will just leave the addresses like
> they are now.
>
> On Thu, Sep 13, 2018 at 10:42 AM mangokm40 
> wrote:
>
>> Mr. Rasmussen,
>>
>> Thanks for the offer!  I definitely need help. :)
>>
>> I looked, and failed to find, the layer without unit#.  That's what I
>> thought would be preferred.  Thanks
>> for spotting it.  I don't see why we would search for a specific unit on
>> a map.  When I navigate, I would
>> just want directions to "1234 NW 33rd Ct", not "1234 NW 33rd Ct Apt 6".
>> If you know
>> of a good reason for the unit #s, let me know.  It doesn't matter to me,
>> since I don't need navigation in
>> Miami.  :D
>> If you think 600k points is big, imagine the building footprints. :)
>>  It's available, if required.  Heck, they
>> even make a 3d building layer available.  But I don't think we'll enjoy
>> the size.
>>
>> I knew the license is not a problem.  However, I saved that email just in
>> case it comes up.
>>
>> I'm going to read the info Mr. Juhász provided.  Unfortunately, I'm way
>> behind here.
>> Also, I replied to all b/c I think that's what I'm supposed to do.
>> However, I don't want to 'bug' people on
>> the list.  Hopefully, someone will let me know if this needs to go
>> off-list.
>>
>>
>> (_8'()
>>
>>
>> ‐‐‐ Original Message ‐‐‐
>> On Wednesday, September 12, 2018 4:17 PM, Leif Rasmussen <
>> 354...@gmail.com> wrote:
>>
>> Hi Mango,
>> I have quite a lot of experience with address imports, and would love to
>> help with Miami.  I have visited Miami several times, and have grown a
>> liking for it.  Adding addresses there would be a real pleasure.
>> There appears to be two address data sets - one with "addr:unit", and one
>> without.  The one with "addr:unit" addresses
>> 
>>  has 1,166,445
>> points, and the one without
>> 
>> has 586,171 points.  Both of these should be considered.  I would suggest
>> importing the one with condos, or "addr:unit" features if the quality is
>> good.  Otherwise, I think that the dataset without addr:unit should be
>> imported.
>> Also, the license seems OK.  According to the Miami-Dade County
>> Buildings Import
>> ,
>> the license is public domain, which they claim is true of all government
>> produced data in Florida.
>> The only issue I see with the data is the size.  My laptop took 5 minutes
>> to open the address points (including addr:unit, so 1,166,445 nodes) and
>> more than 20 minutes to edit a single key.  This could be worked around,
>> though, by splitting up the data.
>> I created a wiki page for the import
>> ,
>> which is a step of the Import Guidelines
>> . Sending a
>> proposal to the local community and imports mailing list will also be
>> needed.
>> I hope that this import will end up working out!
>> Leif Rasmussen
>>
>>
>>
>>
>> (_8'()
>>
>> Sent with ProtonMail  Secure Email.
>>
>> ‐‐‐ Original Message ‐‐‐
>>
> On Wednesday, September 12, 2018 4:17 PM, Leif Rasmussen