Re: [HOT] Building Detection using Machine Learning

[Blake Girardot HOT/OSM]

On Thu, Jan 5, 2017 at 8:43 PM, Stefan Keller  wrote:

>But even if would be 99.8 percent it's important for OSM
> that a human integrates the data into the database.

Just for the record, I agree with this totally. I only see a scenario
where any automatically identified feature would have a human look at
it and the imagery, correct anything as needed and then upload it to
osm.

Cheers
blake

Blake Girardot
Humanitarian OpenStreetMap Team, TM3 Project Manager
skype: jblakegirardot
HOT Core Team Contact: i...@hotosm.org
Live OSM Mapper-Support channel - https://hotosm-slack.herokuapp.com/
BE A PART OF HOT'S MICRO GRANTS: https://donate.hotosm.org/

___
HOT mailing list
HOT@openstreetmap.org
https://lists.openstreetmap.org/listinfo/hot

Re: [HOT] Building Detection using Machine Learning

[Stefan Keller]

Hi Philip

On Mon, Dec 19, 2016 at 3:17 PM, Philip Hunt  wrote:
> I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event a few 
> months ago
> and was interested to see how successful machine learning would be at 
> detecting buildings in
> satellite images. The results look promising but I wanted to know if it could 
> be useful to the
> community and if it’s worth pursuing further.

Looks nice!

Extracting buildings from satellite imagery has been approached quite
some times before. I think that object recognition algos. now get
close to 98%. But even if would be 99.8 percent it's important for OSM
that a human integrates the data into the database.

Your project is very similar to my OSMDeepOD project [1]. We use OSM
and Bing satellite imagery which we feed to the open source software
library TensorFlow with a retrained inception V3 convolution neural
network ("deep learning").

We concentrated on missing crosswalks and are now switching to
building outline, roof orientation and building levels.

As proposed by Rory we take the output and hand it over to MapRoulette
[2]. Actually there are currently 14 MapRoulette challenges ongoing
like e.g. the "Missing Crosswalks of Lucerne in Switzerland". Even if
you don't read German, you can see the progress here [3].

I'm keen to see how your Viola+Jones/Haar-Cascade algorithm performs
versus our deep learning!

We're open to collaborate and accept GitHub pull requests.

:Stefan


[1] https://github.com/geometalab/OSMDeepOD
[2] http://maproulette.org/
[3] http://zebrastreifen-safari.osm.ch/

2017-01-05 16:58 GMT+01:00 Denis Carriere :
> I agree with Rory with the rotated buildings.
>
> However, these building outlines would be great to detect missing buildings
> within an area. It's usually very hard to find "missing" buildings when your
> HOT Tasking Manager is at 100% completed.
>
> This workflow/tool can definitely be used in the validation process of a HOT
> task or adding buildings to an entire city by comparing the existing OSM
> data with the machine learning buildings and find the ones that don't
> overlap.
>
> I don't ever see this ever being imported, but it's a great tool and you
> should continue developing this!
>
> Cheers,
>
> ~~
> Denis Carriere
> GIS Software & Systems Specialist
>
> On Thu, Jan 5, 2017 at 10:41 AM, Rory McCann  wrote:
>>
>> You've noticed how your algoritm isn't able to get properly rotated
>> buildings. And this might be an advantage! All buildings being
>> non-rotated is *obviously* incorrect, so people aren't gonna want to
>> import them, they'll realise that they have to have a human to
>> review/correct it.
>>
>> Perhaps you should load all your "buildings" into MapRoullette or ToFix
>> and people can hope from one spot to the next, mapping buildings? Your
>> "buildings" will ensure that some of the harder work is done, and people
>> can map much faster. It also provides a good measure of how many
>> buildings are in an area, and hence whether the OSM building mapping is
>> complete or some is missing.
>>
>>
>> On 19/12/16 15:17, Philip Hunt wrote:
>> > Hi all,
>> >
>> > I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event
>> > a few months ago and was interested to see how successful machine learning
>> > would be at detecting buildings in satellite images. The results look
>> > promising but I wanted to know if it could be useful to the community and 
>> > if
>> > it’s worth pursuing further. I thought I would post a sample of the results
>> > and then quickly explain the process and issues.
>> >
>> >
>> > Results
>> > ———
>> >
>> > These are the results of a test I ran on project 2101 (Rongo, Kenya -
>> > PMI/USAID) on 1 November 2016. These images show the buildings detected by
>> > the algorithm on the first six unstarted tasks from the project. Potential
>> > buildings are marked with green rectangles:
>> >
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
>> >
>> > As you can see the initial results look promising - most of the
>> > buildings have been detected and the false positive rate is pretty low.
>> >
>> >
>> > Process
>> > 
>> >
>> > I’ve been using the Viola–Jones machine learning algorithm, which
>> > requires training to know what is and isn’t a building. Once the algorithm
>> > is trained, it can be used to detect buildings in new images in a few
>> > seconds.
>> >
>> > The whole process looks like this:
>> >
>> > - Get the HOT project and task data using the HOT API
>> > - Get the satellite imagery of the area from 

Re: [HOT] Building Detection using Machine Learning

[Denis Carriere]

I agree with Rory with the rotated buildings.

However, these building outlines would be great to detect missing buildings
within an area. It's usually very hard to find "missing" buildings when
your HOT Tasking Manager is at 100% completed.

This workflow/tool can definitely be used in the *validation* process of a
HOT task or adding buildings to an entire city by comparing the existing
OSM data with the machine learning buildings and find the ones that don't
overlap.

I don't ever see this ever being imported, but it's a great tool and you
should continue developing this!

Cheers,

*~~*
*Denis Carriere*
*GIS Software & Systems Specialist*

On Thu, Jan 5, 2017 at 10:41 AM, Rory McCann  wrote:

> You've noticed how your algoritm isn't able to get properly rotated
> buildings. And this might be an advantage! All buildings being
> non-rotated is *obviously* incorrect, so people aren't gonna want to
> import them, they'll realise that they have to have a human to
> review/correct it.
>
> Perhaps you should load all your "buildings" into MapRoullette or ToFix
> and people can hope from one spot to the next, mapping buildings? Your
> "buildings" will ensure that some of the harder work is done, and people
> can map much faster. It also provides a good measure of how many
> buildings are in an area, and hence whether the OSM building mapping is
> complete or some is missing.
>
>
> On 19/12/16 15:17, Philip Hunt wrote:
> > Hi all,
> >
> > I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event
> a few months ago and was interested to see how successful machine learning
> would be at detecting buildings in satellite images. The results look
> promising but I wanted to know if it could be useful to the community and
> if it’s worth pursuing further. I thought I would post a sample of the
> results and then quickly explain the process and issues.
> >
> >
> > Results
> > ———
> >
> > These are the results of a test I ran on project 2101 (Rongo, Kenya -
> PMI/USAID) on 1 November 2016. These images show the buildings detected by
> the algorithm on the first six unstarted tasks from the project. Potential
> buildings are marked with green rectangles:
> >
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
> >
> > As you can see the initial results look promising - most of the
> buildings have been detected and the false positive rate is pretty low.
> >
> >
> > Process
> > 
> >
> > I’ve been using the Viola–Jones machine learning algorithm, which
> requires training to know what is and isn’t a building. Once the algorithm
> is trained, it can be used to detect buildings in new images in a few
> seconds.
> >
> > The whole process looks like this:
> >
> > - Get the HOT project and task data using the HOT API
> > - Get the satellite imagery of the area from OSM
> > - Get the nearby existing buildings from the OSM API
> > - Find the existing buildings in the satellite imagery and use these to
> train the algorithm
> > - Run through each incomplete task in the HOT project and detect
> buildings
> > - Output the results as OSM XML
> > - Load the output into JOSM, validate and upload to OSM
> >
> >
> > Issues
> > ———
> >
> > I loaded the output of the algorithm into JOSM and completed tasks 1 and
> 2 of project 2101. However it still took a bit of work to make sure the
> data is good enough for OSM and I think an experienced mapper would have
> taken roughly the same amount of time starting from scratch.
> >
> > The main issue is the algorithm can’t rotate the detected rectangle to
> fit the building shape (as you can see from the example images above, none
> of the rectangles are rotated). I’ve tried using methods such as line
> detection to detect the building and rotate and crop the rectangle around
> the edges - this worked well some of the time and other times went horribly
> wrong.
> >
> > The second issue is false positives. While the examples above we’re
> generally clean, sometimes the algorithm would think a field was a
> building. Because data uploaded to OSM needs to be accurate it can take
> some time checking each potential building in JOSM.
> >
> > Another potential issue could be training samples. When testing I
> trained a new algorithm for each project, using local existing building
> data from OSM as training data. The assumption here is that nearby
> buildings will look like buildings in the project area and that nearby
> building data is available and accurate.
> >
> >
> > Next Steps
> > —
> >
> > The Viola–Jones objection detection research paper was first published
> in 2001 so the 

Re: [HOT] Building Detection using Machine Learning

[Rory McCann]

You've noticed how your algoritm isn't able to get properly rotated
buildings. And this might be an advantage! All buildings being
non-rotated is *obviously* incorrect, so people aren't gonna want to
import them, they'll realise that they have to have a human to
review/correct it.

Perhaps you should load all your "buildings" into MapRoullette or ToFix
and people can hope from one spot to the next, mapping buildings? Your
"buildings" will ensure that some of the harder work is done, and people
can map much faster. It also provides a good measure of how many
buildings are in an area, and hence whether the OSM building mapping is
complete or some is missing.


On 19/12/16 15:17, Philip Hunt wrote:
> Hi all,
> 
> I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event a few 
> months ago and was interested to see how successful machine learning would be 
> at detecting buildings in satellite images. The results look promising but I 
> wanted to know if it could be useful to the community and if it’s worth 
> pursuing further. I thought I would post a sample of the results and then 
> quickly explain the process and issues.
> 
> 
> Results
> ———
> 
> These are the results of a test I ran on project 2101 (Rongo, Kenya - 
> PMI/USAID) on 1 November 2016. These images show the buildings detected by 
> the algorithm on the first six unstarted tasks from the project. Potential 
> buildings are marked with green rectangles:
> 
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
> 
> As you can see the initial results look promising - most of the buildings 
> have been detected and the false positive rate is pretty low.
> 
> 
> Process
> 
> 
> I’ve been using the Viola–Jones machine learning algorithm, which requires 
> training to know what is and isn’t a building. Once the algorithm is trained, 
> it can be used to detect buildings in new images in a few seconds.
> 
> The whole process looks like this:
> 
> - Get the HOT project and task data using the HOT API
> - Get the satellite imagery of the area from OSM
> - Get the nearby existing buildings from the OSM API
> - Find the existing buildings in the satellite imagery and use these to train 
> the algorithm
> - Run through each incomplete task in the HOT project and detect buildings
> - Output the results as OSM XML
> - Load the output into JOSM, validate and upload to OSM
> 
> 
> Issues
> ———
> 
> I loaded the output of the algorithm into JOSM and completed tasks 1 and 2 of 
> project 2101. However it still took a bit of work to make sure the data is 
> good enough for OSM and I think an experienced mapper would have taken 
> roughly the same amount of time starting from scratch.
> 
> The main issue is the algorithm can’t rotate the detected rectangle to fit 
> the building shape (as you can see from the example images above, none of the 
> rectangles are rotated). I’ve tried using methods such as line detection to 
> detect the building and rotate and crop the rectangle around the edges - this 
> worked well some of the time and other times went horribly wrong. 
> 
> The second issue is false positives. While the examples above we’re generally 
> clean, sometimes the algorithm would think a field was a building. Because 
> data uploaded to OSM needs to be accurate it can take some time checking each 
> potential building in JOSM.
> 
> Another potential issue could be training samples. When testing I trained a 
> new algorithm for each project, using local existing building data from OSM 
> as training data. The assumption here is that nearby buildings will look like 
> buildings in the project area and that nearby building data is available and 
> accurate.
> 
> 
> Next Steps
> —
> 
> The Viola–Jones objection detection research paper was first published in 
> 2001 so the algorithm has been around for a while. Machine learning has 
> improved since then and neural networks are showing a lot of promise - using 
> these could increase the reliability and also allow the detected rectangle to 
> be fixed around the edge of the building - meaning a lot less editing in 
> JOSM. I’m also aware of similar projects, but I haven’t found anything that’s 
> able to detect buildings or ready for use yet:
> 
> https://github.com/trailbehind/DeepOSM (find misconfigured roads in OSM)
> https://github.com/patrick-dd/landsat-landstats (predicts population size)
> https://github.com/larsroemheld/OSM-HOT-ConvNet
> 
> 
> I'm not suggesting this could replace volunteers (since algorithms will never 
> be completely accurate), but maybe this could help speed things up or be used 
> to quickly estimate 

Re: [HOT] Building Detection using Machine Learning

[john whelan]

Currently my thoughts run along the lines of use it to identify areas with
buildings.  Same as Mapswipe?, heavens things get outdated so quickly these
days and Mapswipe hasn't been around for that long.

Secondly if you map them but tag them with something like
possiblebuilding=yes then they wouldn't show on a render but could be
selected in JOSM with a todo list plugin and switched to building=yes on
inspection. I think that would meet the workflow side of things that
Fredrick suggested in osm-talk.

Imports, automated edits and image detection software do not have a history
of being problem free in OSM and I think you have to take that into account.

Cheerio John

On 23 December 2016 at 17:46, Robert Banick  wrote:

> Hi Phillip,
>
> I think Andrew, John, etc. have well covered the practical relevance to
> OSM. On a technical note you may consider looking into what the Facebook
> data team is doing: https://code.facebook.com/posts/1676452492623525/
> connecting-the-world-with-better-maps/
>
> I believe a lot of it is based on building detection via machine learning.
>
> Robert
>
> On Wed, Dec 21, 2016 at 7:07 PM Andrew Buck 
> wrote:
>
>> I agree with what the others have said so far... this probably won't be
>> used to feed into the main database but could have other uses.
>>
>> One place where it could be good is in very sparsely populated areas
>> like northern Africa, where there are huge areas of empty space with a
>> few isolated settlements.  In the past we used micro tasking to have
>> users quickly scan though and find village locations and then manually
>> mapped the buildings using the microtask results to find the villages to
>> map.  This worked very well and your algorithm looks robust enough to
>> find the majority of villages.  Also in that case false positives are no
>> problem at all as the mapper just won't map anything when they look
>> there and false negatives are not such a big deal (as long as you
>> identify at least one building in the village the mapper can map the
>> rest themselves anyway).
>>
>> Lastly, this could be used to re-check already mapped villages to see if
>> they need to be revisited to map new buildings, etc.  You could scan an
>> area and for each village spit out a set of numbers of how many
>> buildings are in both datasets, how many are only in yours, and how many
>> are only in OSM.  Based on that we can prioritize areas to be re-looked
>> at by human mappers to update to the new imagery.
>>
>> So all in all this is a very useful tool to have in our toolbox, but
>> just not for direct inclusion into the main database.
>>
>> -AndrewBuck
>>
>>
>>
>>
>> On 12/19/2016 08:17 AM, Philip Hunt wrote:
>> > Hi all,
>> >
>> > I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event
>> a few months ago and was interested to see how successful machine learning
>> would be at detecting buildings in satellite images. The results look
>> promising but I wanted to know if it could be useful to the community and
>> if it’s worth pursuing further. I thought I would post a sample of the
>> results and then quickly explain the process and issues.
>> >
>> >
>> > Results
>> > ———
>> >
>> > These are the results of a test I ran on project 2101 (Rongo, Kenya -
>> PMI/USAID) on 1 November 2016. These images show the buildings detected by
>> the algorithm on the first six unstarted tasks from the project. Potential
>> buildings are marked with green rectangles:
>> >
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
>> >
>> > As you can see the initial results look promising - most of the
>> buildings have been detected and the false positive rate is pretty low.
>> >
>> >
>> > Process
>> > 
>> >
>> > I’ve been using the Viola–Jones machine learning algorithm, which
>> requires training to know what is and isn’t a building. Once the algorithm
>> is trained, it can be used to detect buildings in new images in a few
>> seconds.
>> >
>> > The whole process looks like this:
>> >
>> > - Get the HOT project and task data using the HOT API
>> > - Get the satellite imagery of the area from OSM
>> > - Get the nearby existing buildings from the OSM API
>> > - Find the existing buildings in the satellite imagery and use these to
>> train the algorithm
>> > - Run through each incomplete task in the HOT project and detect
>> buildings
>> > - Output the results as OSM XML
>> > - Load the output into JOSM, validate and upload to OSM
>> >
>> >
>> > Issues
>> > ———
>> >
>> > I loaded the output of the algorithm into JOSM and completed tasks 1
>> and 2 of project 

Re: [HOT] Building Detection using Machine Learning

[Robert Banick]

Hi Phillip,

I think Andrew, John, etc. have well covered the practical relevance to
OSM. On a technical note you may consider looking into what the Facebook
data team is doing:
https://code.facebook.com/posts/1676452492623525/connecting-the-world-with-better-maps/

I believe a lot of it is based on building detection via machine learning.

Robert

On Wed, Dec 21, 2016 at 7:07 PM Andrew Buck  wrote:

> I agree with what the others have said so far... this probably won't be
> used to feed into the main database but could have other uses.
>
> One place where it could be good is in very sparsely populated areas
> like northern Africa, where there are huge areas of empty space with a
> few isolated settlements.  In the past we used micro tasking to have
> users quickly scan though and find village locations and then manually
> mapped the buildings using the microtask results to find the villages to
> map.  This worked very well and your algorithm looks robust enough to
> find the majority of villages.  Also in that case false positives are no
> problem at all as the mapper just won't map anything when they look
> there and false negatives are not such a big deal (as long as you
> identify at least one building in the village the mapper can map the
> rest themselves anyway).
>
> Lastly, this could be used to re-check already mapped villages to see if
> they need to be revisited to map new buildings, etc.  You could scan an
> area and for each village spit out a set of numbers of how many
> buildings are in both datasets, how many are only in yours, and how many
> are only in OSM.  Based on that we can prioritize areas to be re-looked
> at by human mappers to update to the new imagery.
>
> So all in all this is a very useful tool to have in our toolbox, but
> just not for direct inclusion into the main database.
>
> -AndrewBuck
>
>
>
>
> On 12/19/2016 08:17 AM, Philip Hunt wrote:
> > Hi all,
> >
> > I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event
> a few months ago and was interested to see how successful machine learning
> would be at detecting buildings in satellite images. The results look
> promising but I wanted to know if it could be useful to the community and
> if it’s worth pursuing further. I thought I would post a sample of the
> results and then quickly explain the process and issues.
> >
> >
> > Results
> > ———
> >
> > These are the results of a test I ran on project 2101 (Rongo, Kenya -
> PMI/USAID) on 1 November 2016. These images show the buildings detected by
> the algorithm on the first six unstarted tasks from the project. Potential
> buildings are marked with green rectangles:
> >
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
> >
> > As you can see the initial results look promising - most of the
> buildings have been detected and the false positive rate is pretty low.
> >
> >
> > Process
> > 
> >
> > I’ve been using the Viola–Jones machine learning algorithm, which
> requires training to know what is and isn’t a building. Once the algorithm
> is trained, it can be used to detect buildings in new images in a few
> seconds.
> >
> > The whole process looks like this:
> >
> > - Get the HOT project and task data using the HOT API
> > - Get the satellite imagery of the area from OSM
> > - Get the nearby existing buildings from the OSM API
> > - Find the existing buildings in the satellite imagery and use these to
> train the algorithm
> > - Run through each incomplete task in the HOT project and detect
> buildings
> > - Output the results as OSM XML
> > - Load the output into JOSM, validate and upload to OSM
> >
> >
> > Issues
> > ———
> >
> > I loaded the output of the algorithm into JOSM and completed tasks 1 and
> 2 of project 2101. However it still took a bit of work to make sure the
> data is good enough for OSM and I think an experienced mapper would have
> taken roughly the same amount of time starting from scratch.
> >
> > The main issue is the algorithm can’t rotate the detected rectangle to
> fit the building shape (as you can see from the example images above, none
> of the rectangles are rotated). I’ve tried using methods such as line
> detection to detect the building and rotate and crop the rectangle around
> the edges - this worked well some of the time and other times went horribly
> wrong.
> >
> > The second issue is false positives. While the examples above we’re
> generally clean, sometimes the algorithm would think a field was a
> building. Because data uploaded to OSM needs to be accurate it can take
> some time checking each potential building in 

Re: [HOT] Building Detection using Machine Learning

[Andrew Buck]

I agree with what the others have said so far... this probably won't be
used to feed into the main database but could have other uses.

One place where it could be good is in very sparsely populated areas
like northern Africa, where there are huge areas of empty space with a
few isolated settlements.  In the past we used micro tasking to have
users quickly scan though and find village locations and then manually
mapped the buildings using the microtask results to find the villages to
map.  This worked very well and your algorithm looks robust enough to
find the majority of villages.  Also in that case false positives are no
problem at all as the mapper just won't map anything when they look
there and false negatives are not such a big deal (as long as you
identify at least one building in the village the mapper can map the
rest themselves anyway).

Lastly, this could be used to re-check already mapped villages to see if
they need to be revisited to map new buildings, etc.  You could scan an
area and for each village spit out a set of numbers of how many
buildings are in both datasets, how many are only in yours, and how many
are only in OSM.  Based on that we can prioritize areas to be re-looked
at by human mappers to update to the new imagery.

So all in all this is a very useful tool to have in our toolbox, but
just not for direct inclusion into the main database.

-AndrewBuck




On 12/19/2016 08:17 AM, Philip Hunt wrote:
> Hi all,
> 
> I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event a few 
> months ago and was interested to see how successful machine learning would be 
> at detecting buildings in satellite images. The results look promising but I 
> wanted to know if it could be useful to the community and if it’s worth 
> pursuing further. I thought I would post a sample of the results and then 
> quickly explain the process and issues.
> 
> 
> Results
> ———
> 
> These are the results of a test I ran on project 2101 (Rongo, Kenya - 
> PMI/USAID) on 1 November 2016. These images show the buildings detected by 
> the algorithm on the first six unstarted tasks from the project. Potential 
> buildings are marked with green rectangles:
> 
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
> 
> As you can see the initial results look promising - most of the buildings 
> have been detected and the false positive rate is pretty low.
> 
> 
> Process
> 
> 
> I’ve been using the Viola–Jones machine learning algorithm, which requires 
> training to know what is and isn’t a building. Once the algorithm is trained, 
> it can be used to detect buildings in new images in a few seconds.
> 
> The whole process looks like this:
> 
> - Get the HOT project and task data using the HOT API
> - Get the satellite imagery of the area from OSM
> - Get the nearby existing buildings from the OSM API
> - Find the existing buildings in the satellite imagery and use these to train 
> the algorithm
> - Run through each incomplete task in the HOT project and detect buildings
> - Output the results as OSM XML
> - Load the output into JOSM, validate and upload to OSM
> 
> 
> Issues
> ———
> 
> I loaded the output of the algorithm into JOSM and completed tasks 1 and 2 of 
> project 2101. However it still took a bit of work to make sure the data is 
> good enough for OSM and I think an experienced mapper would have taken 
> roughly the same amount of time starting from scratch.
> 
> The main issue is the algorithm can’t rotate the detected rectangle to fit 
> the building shape (as you can see from the example images above, none of the 
> rectangles are rotated). I’ve tried using methods such as line detection to 
> detect the building and rotate and crop the rectangle around the edges - this 
> worked well some of the time and other times went horribly wrong. 
> 
> The second issue is false positives. While the examples above we’re generally 
> clean, sometimes the algorithm would think a field was a building. Because 
> data uploaded to OSM needs to be accurate it can take some time checking each 
> potential building in JOSM.
> 
> Another potential issue could be training samples. When testing I trained a 
> new algorithm for each project, using local existing building data from OSM 
> as training data. The assumption here is that nearby buildings will look like 
> buildings in the project area and that nearby building data is available and 
> accurate.
> 
> 
> Next Steps
> —
> 
> The Viola–Jones objection detection research paper was first published in 
> 2001 so the algorithm has been around for a while. Machine learning has 
> improved since 

Re: [HOT] Building Detection using Machine Learning

[Blake Girardot]

Hi Philip,

I think this looks really promising. I have seen a fair amount of
machine learning and automated feature extraction results from a
number of different sources, and your examples look as accurate as
anything I have seen.

AI/Machine Learning/Automated Feature Extraction are really important
and needed and will be a focus for me personally in 2017. Not that I
can do much besides support and encourage and help with grant writing.

Like you said, I do not think it will be able to replace people
putting their eyes on things and mapping, but what computers can
recognize and help us map, we should be taking advantage of so people
can focus their time on the things computers can not help us map.

For sure if you are interested in doing more of it, it is a very
worthwhile pursuit.

Thank you very much for doing this and sending it and please let me
know if you would like to do more and let us see how HOT or one of our
Missing Maps can help.

Cheers,
Blake

On Mon, Dec 19, 2016 at 3:17 PM, Philip Hunt  wrote:
> Hi all,
>
> I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event a few 
> months ago and was interested to see how successful machine learning would be 
> at detecting buildings in satellite images. The results look promising but I 
> wanted to know if it could be useful to the community and if it’s worth 
> pursuing further. I thought I would post a sample of the results and then 
> quickly explain the process and issues.
>
>
> Results
> ———
>
> These are the results of a test I ran on project 2101 (Rongo, Kenya - 
> PMI/USAID) on 1 November 2016. These images show the buildings detected by 
> the algorithm on the first six unstarted tasks from the project. Potential 
> buildings are marked with green rectangles:
>
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
>
> As you can see the initial results look promising - most of the buildings 
> have been detected and the false positive rate is pretty low.
>
>
> Process
> 
>
> I’ve been using the Viola–Jones machine learning algorithm, which requires 
> training to know what is and isn’t a building. Once the algorithm is trained, 
> it can be used to detect buildings in new images in a few seconds.
>
> The whole process looks like this:
>
> - Get the HOT project and task data using the HOT API
> - Get the satellite imagery of the area from OSM
> - Get the nearby existing buildings from the OSM API
> - Find the existing buildings in the satellite imagery and use these to train 
> the algorithm
> - Run through each incomplete task in the HOT project and detect buildings
> - Output the results as OSM XML
> - Load the output into JOSM, validate and upload to OSM
>
>
> Issues
> ———
>
> I loaded the output of the algorithm into JOSM and completed tasks 1 and 2 of 
> project 2101. However it still took a bit of work to make sure the data is 
> good enough for OSM and I think an experienced mapper would have taken 
> roughly the same amount of time starting from scratch.
>
> The main issue is the algorithm can’t rotate the detected rectangle to fit 
> the building shape (as you can see from the example images above, none of the 
> rectangles are rotated). I’ve tried using methods such as line detection to 
> detect the building and rotate and crop the rectangle around the edges - this 
> worked well some of the time and other times went horribly wrong.
>
> The second issue is false positives. While the examples above we’re generally 
> clean, sometimes the algorithm would think a field was a building. Because 
> data uploaded to OSM needs to be accurate it can take some time checking each 
> potential building in JOSM.
>
> Another potential issue could be training samples. When testing I trained a 
> new algorithm for each project, using local existing building data from OSM 
> as training data. The assumption here is that nearby buildings will look like 
> buildings in the project area and that nearby building data is available and 
> accurate.
>
>
> Next Steps
> —
>
> The Viola–Jones objection detection research paper was first published in 
> 2001 so the algorithm has been around for a while. Machine learning has 
> improved since then and neural networks are showing a lot of promise - using 
> these could increase the reliability and also allow the detected rectangle to 
> be fixed around the edge of the building - meaning a lot less editing in 
> JOSM. I’m also aware of similar projects, but I haven’t found anything that’s 
> able to detect buildings or ready for use yet:
>
> https://github.com/trailbehind/DeepOSM (find misconfigured roads in 

Re: [HOT] Building Detection using Machine Learning

[john whelan]

My personal reaction is OpenStreetMap which is the data base behind HOT is
not very open to machine scanning.  The reason being they've seen some
fairly poor results in the past.

If you can set up some sort of workflow where the images are manually
verified that might be more acceptable but its quite tricky confirming the
existence of a building under building=yes and I'm not that sure it would
save much time over a mapper mapping with the JOSM building_tool plug in.
Based on what I've seen whilst these are more accurate than many mapathon
mappers manage they don't map the buildings quite well enough for me.  Yes
the building is within the square but it seems oversized to an extent and
the orientation can be slightly off.  The more accurate the building size
the more useful it is.  Having said that if it could be done into a
separate database it might well satisfy the demand for buildings that some
NGOs have.

Others will have other thoughts.

Cheerio John

On 19 December 2016 at 09:17, Philip Hunt  wrote:

> Hi all,
>
> I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event a
> few months ago and was interested to see how successful machine learning
> would be at detecting buildings in satellite images. The results look
> promising but I wanted to know if it could be useful to the community and
> if it’s worth pursuing further. I thought I would post a sample of the
> results and then quickly explain the process and issues.
>
>
> Results
> ———
>
> These are the results of a test I ran on project 2101 (Rongo, Kenya -
> PMI/USAID) on 1 November 2016. These images show the buildings detected by
> the algorithm on the first six unstarted tasks from the project. Potential
> buildings are marked with green rectangles:
>
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
> https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
>
> As you can see the initial results look promising - most of the buildings
> have been detected and the false positive rate is pretty low.
>
>
> Process
> 
>
> I’ve been using the Viola–Jones machine learning algorithm, which requires
> training to know what is and isn’t a building. Once the algorithm is
> trained, it can be used to detect buildings in new images in a few seconds.
>
> The whole process looks like this:
>
> - Get the HOT project and task data using the HOT API
> - Get the satellite imagery of the area from OSM
> - Get the nearby existing buildings from the OSM API
> - Find the existing buildings in the satellite imagery and use these to
> train the algorithm
> - Run through each incomplete task in the HOT project and detect buildings
> - Output the results as OSM XML
> - Load the output into JOSM, validate and upload to OSM
>
>
> Issues
> ———
>
> I loaded the output of the algorithm into JOSM and completed tasks 1 and 2
> of project 2101. However it still took a bit of work to make sure the data
> is good enough for OSM and I think an experienced mapper would have taken
> roughly the same amount of time starting from scratch.
>
> The main issue is the algorithm can’t rotate the detected rectangle to fit
> the building shape (as you can see from the example images above, none of
> the rectangles are rotated). I’ve tried using methods such as line
> detection to detect the building and rotate and crop the rectangle around
> the edges - this worked well some of the time and other times went horribly
> wrong.
>
> The second issue is false positives. While the examples above we’re
> generally clean, sometimes the algorithm would think a field was a
> building. Because data uploaded to OSM needs to be accurate it can take
> some time checking each potential building in JOSM.
>
> Another potential issue could be training samples. When testing I trained
> a new algorithm for each project, using local existing building data from
> OSM as training data. The assumption here is that nearby buildings will
> look like buildings in the project area and that nearby building data is
> available and accurate.
>
>
> Next Steps
> —
>
> The Viola–Jones objection detection research paper was first published in
> 2001 so the algorithm has been around for a while. Machine learning has
> improved since then and neural networks are showing a lot of promise -
> using these could increase the reliability and also allow the detected
> rectangle to be fixed around the edge of the building - meaning a lot less
> editing in JOSM. I’m also aware of similar projects, but I haven’t found
> anything that’s able to detect buildings or ready for use yet:
>
> https://github.com/trailbehind/DeepOSM (find misconfigured roads in OSM)
>