Re: [Talk-GB] Importing Website Data

[Paul Gillard]

Being a lone mapper up to now it's been interesting to read everyone's
comments, a good wide range of views.

Brian, to answer your question about priorities, I don't particularly have
any. My thoughts were to choose a selection of different types of company
and if see we can't work out where the low hanging fruit is based on the
responses we get. I'm open to other approaches though.

Rob Nickerson has been in touch about the help that the OSM UK chapter
could offer and we'll get a plan together for how to approach companies. If
anyone has thoughts to add on this then they're very welcome.

I'll come back if and when I get anywhere.

Cheers

On 19 December 2017 at 14:46, Brian Prangle  wrote:

> Paul thank you for suggesting this, it's certainly something as a UK
> community (and I guess more widely)we need to deal with. Unfortunately
> website data can be problematic, as others have already indicated, for us
> to use, but instead we should ask the organisations concerned to provide
> the data in a compatible format. Now that we have a formal body
> incorporated as the UK Local Chapter that should be a good vehicle for
> making the requests. Do you have an suggestions for a priority list? Does
> anyone else have any priorities?
>
> For those who decry the approach of using third party data, preferring
> instead the personally surveyed approach, I echo Ilya's and Warin's
> sentiments: Lloyds TSB demerged in 2013 and we still have 200 instances of
> Lloyds TSB, TA Centres became Army Reserve Centres at about the same time
> and we  still have about 40 instances of TA Centre, Shell purchased Total
> filling stations in 2012, and during the recent validation exercise on
> Shell data, name=Total was the commonest error. What about the the
> wholesale closure and transfer  of Post Offices or the planned closure of
> thousands of BT phone boxes?  We don't have the number of motivated mappers
> to do this,  and expecting evrything to be ground surveyed might be a
> reason why we have such a high attrition rate; so we should emulate the
> rest of society and use automated IT methods to assist us and make our
> lives easier where appropriate. OSM is a balance between IT  imported
> data/automated edits and human ground surveys.
>
> Regards
>
> Brian
>
>
>
> On 18 December 2017 at 13:15,  wrote:
>
>> Hi,
>>
>> Reading all the talk of Walmart and Shell imports recently got me to
>> wondering why we can't be doing more of this kind of thing.
>>
>> If store data can be pulled from directly from a company's public facing
>> website ('store finder' page) is there any reason we can't do such imports
>> without discussion with/permission from the company concerned?
>>
>> Paul
>>
>> ___
>> Talk-GB mailing list
>> Talk-GB@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-gb
>>
>
>
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Andy Mabbett]

On 21 December 2017 at 10:28, Frederik Ramm  wrote:

> Your approach will ultimately lead to every last
> large-chain pub in England being nicely mapped from afar, whereas the
> independent pub next door has to wait until a mapper comes around.

Did we not have a data set supplied by a trade body, that enabled us
to add every independent cycle shop in the UK?

-- 
Andy Mabbett
@pigsonthewing
http://pigsonthewing.org.uk

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Andy Townsend]

On 21/12/2017 09:15, Ilya Zverev wrote:

...
Or do you think the map does not need imports and that every shop and amenity 
will be mapped without them before they are out of business?



Maybe it's worth having a look at an example.

http://www.openstreetmap.org/way/245390956 is a petrol station in South 
Normanton, not that far from me.  It's not that out of the ordinary.  It 
was already mapped in OSM as a Shell petrol station (as most were) 
before Navads came along.


Here's the data that Navads was able to provide:

 * Postcode
 * Phone number


But here's the data that Navads didn't add and was already in OSM

 * There's a Post Office here
 * There's an ATM
 * There's a Spar convenience store
 * There's a branch of WHSmith


Clearly being able to provide e.g. licence-unencumbered postcode data is 
useful, but if we relied _just_ on the likes of Navads we'd have a 
pretty poor view of what's available here.


Similarly, Mark Goodge's point ("Inaccurate data is worse than missing 
data") is a good one, but in this case we do have a source on the 
changeset (see e.g. http://www.openstreetmap.org/way/69974213 up the 
road and http://www.openstreetmap.org/changeset/54785385 ) so if stuff 
is contradicted by future survey it's clear where it came from and can 
be easily updated.


Best Regards

Andy

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Colin Smale]

This discussion brings a couple of similar sayings to mind (and there
are many more in the same vein): 

_To Sacrifice The Good On The Altar Of The Perfect_
= and = 

_Perfect is the enemy of good_ [1] 

A dataset will never be perfect. Resisting an import because a small
proportion of the data is likely to be wrong, is a standpoint that can
never be countered. And yet, a more pragmatic assessment may come to a
different conclusion. Is "OSM" more valuable to its users after the
import than before? Is the net result considered an improvement?

Such assessments require a definition of "OSM", its "value" and its
"users" and I don't think there is a clear consensus about this, only a
collection of opinions. 

--colin 

[1] https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good 

On 2017-12-21 17:06, Mark Goodge wrote:

> On 21/12/2017 15:49, Frederik Ramm wrote: Hi,
> 
> On 21.12.2017 16:13, Mark Goodge wrote: My vision of OSM is a movement which 
> places its users first, by
> providing the maximum utility possible for those who look at the maps.
> That means maximising the quantity, accuracy, relevance and timeliness
> of the data. 
> That is certainly a valid approach that many will subscribe to. The
> goals you mention will sometimes have to be weighed against each other
> ("is a large amount of inaccurate data better than a small amount of
> accurate data", "is it good to add this bulk data which is unlikely to
> be cared for by mappers and hence will soon lack in timeliness" etc) but
> on the whole they're a good selection.

Data has to be accurate. Inaccurate data is worse than missing data
(although, it must be noted that imprecise is not the same as
inaccurate, and we can tolerate a reasonable amount of imprecision
provided it is not misleadingly imprecise).

> I think that relevance plays a big role, and commercial players tend to
> claim that concept for themselves ("tell us something about you so we
> can display ads that are relevant to you") - in my view, a pub in your
> town is not "relevant" because the chain operating it thinks that it
> should be, but because the locals find it relevant.

As far as a map is concerned, something is relevant if it is there. Even
if only one person a year actually wants to know it's there :-)

Mark

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Mark Goodge]

On 21/12/2017 15:49, Frederik Ramm wrote:

Hi,

On 21.12.2017 16:13, Mark Goodge wrote:

My vision of OSM is a movement which places its users first, by
providing the maximum utility possible for those who look at the maps.
That means maximising the quantity, accuracy, relevance and timeliness
of the data.


That is certainly a valid approach that many will subscribe to. The
goals you mention will sometimes have to be weighed against each other
("is a large amount of inaccurate data better than a small amount of
accurate data", "is it good to add this bulk data which is unlikely to
be cared for by mappers and hence will soon lack in timeliness" etc) but
on the whole they're a good selection.


Data has to be accurate. Inaccurate data is worse than missing data 
(although, it must be noted that imprecise is not the same as 
inaccurate, and we can tolerate a reasonable amount of imprecision 
provided it is not misleadingly imprecise).



I think that relevance plays a big role, and commercial players tend to
claim that concept for themselves ("tell us something about you so we
can display ads that are relevant to you") - in my view, a pub in your
town is not "relevant" because the chain operating it thinks that it
should be, but because the locals find it relevant.


As far as a map is concerned, something is relevant if it is there. Even 
if only one person a year actually wants to know it's there :-)


Mark

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Frederik Ramm]

Hi,

On 21.12.2017 16:13, Mark Goodge wrote:
> My vision of OSM is a movement which places its users first, by
> providing the maximum utility possible for those who look at the maps.
> That means maximising the quantity, accuracy, relevance and timeliness
> of the data.

That is certainly a valid approach that many will subscribe to. The
goals you mention will sometimes have to be weighed against each other
("is a large amount of inaccurate data better than a small amount of
accurate data", "is it good to add this bulk data which is unlikely to
be cared for by mappers and hence will soon lack in timeliness" etc) but
on the whole they're a good selection.

I think that relevance plays a big role, and commercial players tend to
claim that concept for themselves ("tell us something about you so we
can display ads that are relevant to you") - in my view, a pub in your
town is not "relevant" because the chain operating it thinks that it
should be, but because the locals find it relevant.

I do concede that I've mapped many a business in random places I've been
to without asking the locals if it is relevant to them though.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Brian Prangle]

Eloquently put as ever Frederick. I share your lament about the seemingly
unattainable vision, which I happen to share. I would dearly love there to
be 10 times the number of consistently active mappers in the UK, but it
seems to be an order of magnitude more difficult ot persuade people to
maintain a map and "fill in the gaps" rather than build out a map from
scratch. I also share your  view of the increasing grip that large
corporations have on our social and political life, much to our detriment.

However we share a space with them and if they have data we can use then we
should use it: it won't be perfect but it's in the interests of
corporations for their customers and suppliers to be able to find their
locations so they'll be expending some effort to achieve a high quality (
and Tom I've seen inside enough large corporations not be naive about their
level of quality, but as Warin says some data is better than no data  as
long as we  validate it as much as we can.). Also the population at large
who might  use our map will want to find these large corporate outlets, so
we need to make the effort for them.

 I'm concerned that rate of change in the UK's High Streets with closures
of banks, pubs, chain stores,phone boxes and almagamations,takeovers,
brand-name changes will overwhelm us and degrade the reputation we have
laboured to establish. I've certainly lost the will to keep up with this
consistently.  I'm almost convinced by Frederick's argument that we
shouldn't map any of this, but just the base information of building outine
and address and building name where it is prominent, just as Ordnance
Survey does. In mapping this level of detail we seem to have made  a rod
for our own backs.

Regards

Brian

On 21 December 2017 at 10:28, Frederik Ramm  wrote:

> Hi,
>
> On 21.12.2017 10:15, Ilya Zverev wrote:
> > Frederik and Tom, please explain what has been wrong with the last
> import, and why osm_conflate + cf_audit tools used for it (conflation +
> community validation) still do not attain the required quality for OSM
> contributions?
>
> Your tools are certainly best on the market. Still I wish they had not
> been written at all.
>
> > How would you build a process for importing large batches of business
> chains?
>
> I am 100% against importing large batches of business chains. That data
> has no place in *my* vision of OSM; it should reside in a separate open
> database which anyone can choose to add to their local data set if and
> when needed.
>
> My vision of OSM is that of a grassroots movement that makes its own
> decisions about the data. We decide what the fuel station around the
> corner is called - not the marketing department of the fuel station
> chain. Corporate interests already dictate enough of our everyday lives;
> I don't want Shell to "help" me show the "right" information about their
> fuel stations.
>
> A lot is wrong with capitalism; large chains are one of those things.
> Small independent shops, booksellers, or pubs *already* face
> difficulties against the marketing and purchasing power of big
> corporations. Your approach will ultimately lead to every last
> large-chain pub in England being nicely mapped from afar, whereas the
> independent pub next door has to wait until a mapper comes around. You
> might say "well one pub mapped in town is better than no pub", but this
> is not my opinion; I would have wished for OpenStreetMap to be a map
> made by the people, not a map made by corporations large enough to hire
> SEO companies to manage their online presence.
>
> I am totally aware that "my" vision of OSM is a romantic dream and that
> ultimately, OSM is going to be just another venue for big money to
> battle it out, but please allow me to at least talk about this romantic
> dream from time to time.
>
> As I said, part of "growing up" is probably to toss your romantic dreams
> an accept reality for what it is. I'm totally ok with you being more
> grown up than me.
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb
>
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Frederik Ramm]

Hi,

On 21.12.2017 10:15, Ilya Zverev wrote:
> Frederik and Tom, please explain what has been wrong with the last import, 
> and why osm_conflate + cf_audit tools used for it (conflation + community 
> validation) still do not attain the required quality for OSM contributions?

Your tools are certainly best on the market. Still I wish they had not
been written at all.

> How would you build a process for importing large batches of business chains? 

I am 100% against importing large batches of business chains. That data
has no place in *my* vision of OSM; it should reside in a separate open
database which anyone can choose to add to their local data set if and
when needed.

My vision of OSM is that of a grassroots movement that makes its own
decisions about the data. We decide what the fuel station around the
corner is called - not the marketing department of the fuel station
chain. Corporate interests already dictate enough of our everyday lives;
I don't want Shell to "help" me show the "right" information about their
fuel stations.

A lot is wrong with capitalism; large chains are one of those things.
Small independent shops, booksellers, or pubs *already* face
difficulties against the marketing and purchasing power of big
corporations. Your approach will ultimately lead to every last
large-chain pub in England being nicely mapped from afar, whereas the
independent pub next door has to wait until a mapper comes around. You
might say "well one pub mapped in town is better than no pub", but this
is not my opinion; I would have wished for OpenStreetMap to be a map
made by the people, not a map made by corporations large enough to hire
SEO companies to manage their online presence.

I am totally aware that "my" vision of OSM is a romantic dream and that
ultimately, OSM is going to be just another venue for big money to
battle it out, but please allow me to at least talk about this romantic
dream from time to time.

As I said, part of "growing up" is probably to toss your romantic dreams
an accept reality for what it is. I'm totally ok with you being more
grown up than me.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Tom Hughes]

On 21/12/17 09:15, Ilya Zverev wrote:


Frederik and Tom, please explain what has been wrong with the last import, and 
why osm_conflate + cf_audit tools used for it (conflation + community 
validation) still do not attain the required quality for OSM contributions?


I wasn't commenting on any particular import just on the general principles.

I was merely trying to point out that the view that people often have of 
"official" data as somehow perfect is often far from the truth. I've 
heard enough real world stories of databases inside companies to know 
just how far from reality it can be.


People often imagine these things as perfectly curated and fully 
normalised and standardised when the reality is often that they're 
maintained as an excel spreadsheet by this weeks intern.



How would you build a process for importing large batches of business chains? 
Can I improve something in my tools, or should I build something better from 
scratch?


Well I probably wouldn't because it doesn't especially interest me and I 
have no commercial reason for wanting to do so. Plus I know that it's an 
extremely hard problem.


It's quite true that an import may well be better than nothing where 
things haven't been mapped or aren't being actively maintained, but it's 
equally true that an import that includes updating existing objects may 
sometimes make things worse, and I don't know how you can tell when you 
are making an object worse.


That's what makes it so hard.

That said I will certainly agree that what you're doing is far better 
than what many companies trying to get their clients locations into OSM 
have done in the past.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Ilya Zverev]

Again, with sarcasm.

Frederik and Tom, please explain what has been wrong with the last import, and 
why osm_conflate + cf_audit tools used for it (conflation + community 
validation) still do not attain the required quality for OSM contributions?

How would you build a process for importing large batches of business chains? 
Can I improve something in my tools, or should I build something better from 
scratch?

Or do you think the map does not need imports and that every shop and amenity 
will be mapped without them before they are out of business?

Ilya

> 21 дек. 2017 г., в 10:20, Frederik Ramm  написал(а):
> 
> Hi,
> 
> On 19.12.2017 16:20, Tom Hughes wrote:
>> Which is exactly what everybody said about OSM when it started - that it
>> couldn't possibly work and there'd never be enough people.
>> Pretty sure we proved them wrong.
> 
> Perhaps that's what people mean when they say "OSM has to grow up" -
> that we need to finally replace our youthful enthusiasm with a more
> resigned "the grown-ups were right after all" attitude.
> 
> Luckily, SEO companies are here to help, to enable even more people to
> "engage with brands" through our map. Win-win!
> 
> Bye
> Frederik
> 
> -- 
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
> 
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb


___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Frederik Ramm]

Hi,

On 19.12.2017 16:20, Tom Hughes wrote:
> Which is exactly what everybody said about OSM when it started - that it
> couldn't possibly work and there'd never be enough people.
> Pretty sure we proved them wrong.

Perhaps that's what people mean when they say "OSM has to grow up" -
that we need to finally replace our youthful enthusiasm with a more
resigned "the grown-ups were right after all" attitude.

Luckily, SEO companies are here to help, to enable even more people to
"engage with brands" through our map. Win-win!

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Ian Caldwell]

On 19 December 2017 at 15:20, Tom Hughes  wrote:

>
>
> The fundamental problem of imports that conflate with existing data is
> that you have way of knowing whether or not you are actually improving
> anything - you are making an assumption that an "official" source will be
> up to date and accurate but in the real world they are often anything but.
>
>
I don't think anybody is suggesting taking imports with out some form of
checking/validation first.  Tools that validate imports are a good idea.

The question should be "does it make the OSM database better".
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Tom Hughes]

On 19/12/17 14:46, Brian Prangle wrote:

For those who decry the approach of using third party data, preferring 
instead the personally surveyed approach, I echo Ilya's and Warin's 
sentiments: Lloyds TSB demerged in 2013 and we still have 200 instances 
of Lloyds TSB, TA Centres became Army Reserve Centres at about the same 
time  and we  still have about 40 instances of TA Centre, Shell 
purchased Total filling stations in 2012, and during the recent 
validation exercise on Shell data, name=Total was the commonest error. 
What about the the wholesale closure and transfer  of Post Offices or 
the planned closure of thousands of BT phone boxes?  We don't have the 
number of motivated mappers to do this,  and expecting evrything to be 
ground surveyed might be a reason why we have such a high attrition 
rate; so we should emulate the rest of society and use automated IT 
methods to assist us and make our lives easier where appropriate. OSM is 
a balance between IT  imported data/automated edits and human ground 
surveys.


Which is exactly what everybody said about OSM when it started - that it 
couldn't possibly work and there'd never be enough people.


Pretty sure we proved them wrong.

The fundamental problem of imports that conflate with existing data is 
that you have way of knowing whether or not you are actually improving 
anything - you are making an assumption that an "official" source will 
be up to date and accurate but in the real world they are often anything 
but.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Brian Prangle]

Paul thank you for suggesting this, it's certainly something as a UK
community (and I guess more widely)we need to deal with. Unfortunately
website data can be problematic, as others have already indicated, for us
to use, but instead we should ask the organisations concerned to provide
the data in a compatible format. Now that we have a formal body
incorporated as the UK Local Chapter that should be a good vehicle for
making the requests. Do you have an suggestions for a priority list? Does
anyone else have any priorities?

For those who decry the approach of using third party data, preferring
instead the personally surveyed approach, I echo Ilya's and Warin's
sentiments: Lloyds TSB demerged in 2013 and we still have 200 instances of
Lloyds TSB, TA Centres became Army Reserve Centres at about the same time
and we  still have about 40 instances of TA Centre, Shell purchased Total
filling stations in 2012, and during the recent validation exercise on
Shell data, name=Total was the commonest error. What about the the
wholesale closure and transfer  of Post Offices or the planned closure of
thousands of BT phone boxes?  We don't have the number of motivated mappers
to do this,  and expecting evrything to be ground surveyed might be a
reason why we have such a high attrition rate; so we should emulate the
rest of society and use automated IT methods to assist us and make our
lives easier where appropriate. OSM is a balance between IT  imported
data/automated edits and human ground surveys.

Regards

Brian



On 18 December 2017 at 13:15,  wrote:

> Hi,
>
> Reading all the talk of Walmart and Shell imports recently got me to
> wondering why we can't be doing more of this kind of thing.
>
> If store data can be pulled from directly from a company's public facing
> website ('store finder' page) is there any reason we can't do such imports
> without discussion with/permission from the company concerned?
>
> Paul
>
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb
>
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Warin]

On 19-Dec-17 05:42 AM, Ilya Zverev wrote:

18.12.2017 19:14, Frederik Ramm пишет:

Hi,

On 12/18/2017 03:09 PM, SK53 wrote:

Personally, I'd also be chary of turning OSM into a repository of
scraped data rather than one of surveyed geodata.


And more: the more imports you do, the more OpenStreetMap becomes an IT
project where computer nerds script, collect, convert, conflate, and
interpolate away, instead of a project where all sorts of people from
all walks of life contribute their knowledge.

I think we're not an IT project, and that's good.


We are partly an IT project, which you cannot take away. Also, the 
number and size of imports does not affect how OSM is perceived.


For example, OSMers map very few POIs (relatively), and after they do, 
they update virtually zero of them. Not counting a few active mappers 
in UK and Germany.


To think that you can import everything is to overstate the quantity 
of open data in the world. Open data is mostly points of interest. In 
some countries it's also roads. And that's all. OSM is so much bigger 
than that.


As for importing POIs, I think that needs to be done, because no 
mapper group would be able to map all fuel station, all Tesco shops, 
or all museums. And if they do, in ten years, but that time half of 
what they've collected will be obsolete. The choice is not between 
manual mapping and importing, it's between importing and not having 
the data ever.


And since there isn't open data on everything, even if you import all 
POIs you can possibly have in machine-readable format, that will still 
be at most 10% of all POIs, even in UK or Germany. Plenty to map by 
going outside.


So even an army of IT folks armed with data scrapers and contracts 
with every data aggregator would be no match to common mappers, armed 
with a pen and a camera. I fail to see what you're afraid of.


Two divergent points of view if you go to the extremes. Here are some 
extremes;



A) Only physically survey data is for OSM.
Not possible to physically survey some things like National Park 
boundaries in Australia -  where they are marked the marks are well 
inside the actual boundary. A physical survey will result in large errors.

B) Only open data is to be used.
Some of this is out of data - an on the ground physical survey will 
confirm it.


The best for me is a mix of sources, with a preference for the on the 
ground physical survey. However I can map a lot more area from open 
source data quickly and accurately that I can from a physical survey. In 
some cases of tree cover or GPS reflections, satellite imagery is more 
accurate that GPS surveys. So a mix of sources for me.


___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Ilya Zverev]

18.12.2017 19:14, Frederik Ramm пишет:

Hi,

On 12/18/2017 03:09 PM, SK53 wrote:

Personally, I'd also be chary of turning OSM into a repository of
scraped data rather than one of surveyed geodata.


And more: the more imports you do, the more OpenStreetMap becomes an IT
project where computer nerds script, collect, convert, conflate, and
interpolate away, instead of a project where all sorts of people from
all walks of life contribute their knowledge.

I think we're not an IT project, and that's good.


We are partly an IT project, which you cannot take away. Also, the 
number and size of imports does not affect how OSM is perceived.


For example, OSMers map very few POIs (relatively), and after they do, 
they update virtually zero of them. Not counting a few active mappers in 
UK and Germany.


To think that you can import everything is to overstate the quantity of 
open data in the world. Open data is mostly points of interest. In some 
countries it's also roads. And that's all. OSM is so much bigger than that.


As for importing POIs, I think that needs to be done, because no mapper 
group would be able to map all fuel station, all Tesco shops, or all 
museums. And if they do, in ten years, but that time half of what 
they've collected will be obsolete. The choice is not between manual 
mapping and importing, it's between importing and not having the data ever.


And since there isn't open data on everything, even if you import all 
POIs you can possibly have in machine-readable format, that will still 
be at most 10% of all POIs, even in UK or Germany. Plenty to map by 
going outside.


So even an army of IT folks armed with data scrapers and contracts with 
every data aggregator would be no match to common mappers, armed with a 
pen and a camera. I fail to see what you're afraid of.


Ilya


___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Frederik Ramm]

Hi,

On 12/18/2017 03:09 PM, SK53 wrote:
> Personally, I'd also be chary of turning OSM into a repository of
> scraped data rather than one of surveyed geodata.

And more: the more imports you do, the more OpenStreetMap becomes an IT
project where computer nerds script, collect, convert, conflate, and
interpolate away, instead of a project where all sorts of people from
all walks of life contribute their knowledge.

I think we're not an IT project, and that's good.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Gregrs]

Hi Paul,

On Mon, Dec 18, 2017 at 01:15:08PM +, paulmgill...@gmail.com wrote:

If store data can be pulled from directly from a company's public 
facing website ('store finder' page) is there any reason we can't do 
such imports without discussion with/permission from the company 
concerned?


I believe the problem is licencing. Just because the data is publicly 
accessible on a company's website doesn't mean that it has been released 
under a licence compatible with that of OpenStreetMap. In practice I 
would imagine that most companies would be pleased to have the locations 
of their stores/branches on OpenStreetMap, but another issue is that 
some of the data might be derived from other sources such as a Royal 
Mail database.


Thanks,
Greg

--
Twitter: @gregrs_uk
http://gregrs.dev.openstreetmap.org
PGP key ID: 64907C8A
Fingerprint: EBD1 077F CCDD 841E A505 3FAA D2E8 592E 6490 7C8A



signature.asc
Description: PGP signature
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Tom Hughes]

On 18/12/17 13:15, paulmgill...@gmail.com wrote:


Reading all the talk of Walmart and Shell imports recently got me to wondering 
why we can't be doing more of this kind of thing.

If store data can be pulled from directly from a company's public facing 
website ('store finder' page) is there any reason we can't do such imports 
without discussion with/permission from the company concerned?


Just a small thing called copyright and/or database right.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Importing Website Data

[Dan S]

Hi,

People have mentioned this before. The conversation usually then goes
to copyright (and the question of whether simple facts (such as
addresses, opening hours etc) can be copyrighted), then in fact it's
database rights that are probably the main barrier.* I'm not central
to all of this, nor a lawyer, but I'd say it's definitely not clear
that it's OK, and thus inappropriate to scrape such information
without explicit permission from the institutions concerned. But then,
getting permission, checking the data, taking it step by step, seems
like a positive thing to be doing. Just my 2p though!

Dan


* see eg https://www.out-law.com/page-5698

2017-12-18 13:15 GMT+00:00  :
> Hi,
>
> Reading all the talk of Walmart and Shell imports recently got me to 
> wondering why we can't be doing more of this kind of thing.
>
> If store data can be pulled from directly from a company's public facing 
> website ('store finder' page) is there any reason we can't do such imports 
> without discussion with/permission from the company concerned?
>
> Paul
>
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

[Talk-GB] Importing Website Data

[paulmgillard]

Hi,

Reading all the talk of Walmart and Shell imports recently got me to wondering 
why we can't be doing more of this kind of thing.

If store data can be pulled from directly from a company's public facing 
website ('store finder' page) is there any reason we can't do such imports 
without discussion with/permission from the company concerned? 

Paul

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb