Re: [Imports] Worldwide fuel stations import, 59k objects

[Michael Reichert]

Hi Ilya,

Am 07.03.2018 um 18:27 schrieb Ilya Zverev:
> Here is the GeoJSON file for the import (42 MB — don't try to open it in 
> geojson.io, use QGIS): https://transfer.sh/12oTYj/fuel.json

This file is not accessible any more. Could you please provide a more
stable link (you are running a web server for your Audit tool, don't
you?) to ease the review without the limitations of your Conflator tool.

Best regards

Michael


-- 
Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten
ausgenommen)
I prefer GPG encryption of emails. (does not apply on mailing lists)



signature.asc
Description: OpenPGP digital signature
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[James]

why is everyone complaining about qgis, it opens in JOSM too with the
geojson plugin.

On Mar 8, 2018 2:55 PM, "Andy Townsend"  wrote:

> On 08/03/2018 17:31, Clifford Snow wrote:
>
>
> Andy,
> Ilya did provide a geojson [1] of the entire proposed import. If you look
> back in the thread, IIya replied to my request. While large, it opens in
> QGIS nicely.
>
> [1]  https://transfer.sh/12oTYj/fuel.json
>
>
> Indeed, but that's not really very helpful to the casual reviewer - we
> can't seriously be expecting every local mapper to install a copy of QGIS
> just to check a proposed import can we? That's very much in "in the bottom
> of a locked filing cabinet stuck in a disused lavatory with a sign on the
> door saying 'Beware of the Leopard'" territory*.  Also just looking at the
> data like that means there's no way to verify that a particular point looks
> OK or has problems.
>
> Actually, when the data does eventually load*** it's clear that there are
> some serious problems with it.  To take a local example previously
> mentioned when the Shell import was discussed
> https://www.openstreetmap.org/node/298006841 - there appears to be a
> proposal to change the opening hours there from the "Shell" values to
> Navads' "24/7".  It might have changed of course, but if so it's within the
> last month or so.  Another example is https://www.openstreetmap.org/
> node/280943195 - the brand in OSM is correct, the proposed brand is
> wrong, as is the proposed website.
>
> Perhaps like the initial attempt with the Shell data Zverik needs to go
> away and then come back once it has been corrected at source?  That's
> essentially what happened with the previous Shell UK import** - at the
> second attempt a significant number of obvious errors (discussed on this
> list) had been corrected.  In addition I'd suggest that this import should
> not automatically overwrite data that has an object source or a changeset
> source of something containing "survey", local_knowledge" or similar.
>
> It also does not make sense to suggest that an "advertising" company
> update objects in OSM on their whim in the future, especially given the
> poor quality of the data they're trying to import now.  It's not just the
> UK either - their have been quite a few "Zverik, this one looks wrong"
> comments on IRC from other countries too.
>
> Best Regards,
>
> Andy
>
>
> * 40 years to the day since first broadcast, incidentally:
> https://en.wikipedia.org/wiki/The_Hitchhiker's_Guide_to_the_
> Galaxy_Primary_and_Secondary_Phases#Fit_the_First
>
> ** See https://lists.openstreetmap.org/pipermail/imports/2017-
> May/thread.html#4956 and https://lists.openstreetmap.
> org/pipermail/talk-gb/2017-May/thread.html#20205 et al.
>
> *** The website's still broken - I still can't mark data as "valid" or
> "invalid" even when logged in.
>
>
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
>
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Martin Koppenhoefer]

sent from a phone

> On 8. Mar 2018, at 19:51, Christoph Hormann  wrote:
> 
> I understand your point but i have to strongly disagree here.  IMO the 
> English language dominance is one of the biggest structural problems of 
> OSM in total and the biggest long term obstacle in becoming what OSM 
> aims to be - the best map/geodatabase of the world.  And i completely 
> agree this is a very hard problem, maybe even impossible to solve.  But 
> that does not mean we should not try.


I agree with this and your ambassador idea surely merits a try. Still it would 
already be helpful to send an email to the national lists, where an import 
concerns items in the country, even in English would be much better than 
nothing (there could be some threshold there, to me 1000 objects would seem a 
lot, I’d see it more in the 50-100 range). 

Cheers,
Martin 
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Michael Reichert]

Hi Ilya,

Am 07.03.2018 um 18:27 schrieb Ilya Zverev:
> Here is the GeoJSON file for the import (42 MB — don't try to open it in 
> geojson.io, use QGIS): https://transfer.sh/12oTYj/fuel.json

Thank you for offering the GeoJSON file for download. I opened in QGIS
but I am confused about the meaning of the columns. Some columns seem
obvious:

osm_id: OSM object ID
osm_type: node/way/relation
tags.:  is the OSM key and the value of this field is what is
mapped in OSM
tags_news.: tag to be added, is that correct?
tags_changed.: tag to be modified, is that correct?

Could you please add the explanation to the wiki page to be created?

I would like to see some kind of map instead of a boring table because
having to use QGIS is a hurdle (at least to the average mapper whom we
want to participate in a review because he has local knowledge).
However, offering a graphical preview is no strict requirement for an
import. There have been lots of imports which did not have a preview map
but just a shape file (or GeoJSON file).

Best regards

Michael


-- 
Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten
ausgenommen)
I prefer GPG encryption of emails. (does not apply on mailing lists)



signature.asc
Description: OpenPGP digital signature
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Andy Townsend]

On 08/03/2018 17:31, Clifford Snow wrote:


Andy,
Ilya did provide a geojson [1] of the entire proposed import. If you 
look back in the thread, IIya replied to my request. While large, it 
opens in QGIS nicely.


[1] https://transfer.sh/12oTYj/fuel.json


Indeed, but that's not really very helpful to the casual reviewer - we 
can't seriously be expecting every local mapper to install a copy of 
QGIS just to check a proposed import can we? That's very much in "in the 
bottom of a locked filing cabinet stuck in a disused lavatory with a 
sign on the door saying 'Beware of the Leopard'" territory*.  Also just 
looking at the data like that means there's no way to verify that a 
particular point looks OK or has problems.


Actually, when the data does eventually load*** it's clear that there 
are some serious problems with it.  To take a local example previously 
mentioned when the Shell import was discussed 
https://www.openstreetmap.org/node/298006841 - there appears to be a 
proposal to change the opening hours there from the "Shell" values to 
Navads' "24/7".  It might have changed of course, but if so it's within 
the last month or so.  Another example is 
https://www.openstreetmap.org/node/280943195 - the brand in OSM is 
correct, the proposed brand is wrong, as is the proposed website.


Perhaps like the initial attempt with the Shell data Zverik needs to go 
away and then come back once it has been corrected at source? That's 
essentially what happened with the previous Shell UK import** - at the 
second attempt a significant number of obvious errors (discussed on this 
list) had been corrected.  In addition I'd suggest that this import 
should not automatically overwrite data that has an object source or a 
changeset source of something containing "survey", local_knowledge" or 
similar.


It also does not make sense to suggest that an "advertising" company 
update objects in OSM on their whim in the future, especially given the 
poor quality of the data they're trying to import now.  It's not just 
the UK either - their have been quite a few "Zverik, this one looks 
wrong" comments on IRC from other countries too.


Best Regards,

Andy


* 40 years to the day since first broadcast, incidentally: 
https://en.wikipedia.org/wiki/The_Hitchhiker's_Guide_to_the_Galaxy_Primary_and_Secondary_Phases#Fit_the_First


** See 
https://lists.openstreetmap.org/pipermail/imports/2017-May/thread.html#4956 
and 
https://lists.openstreetmap.org/pipermail/talk-gb/2017-May/thread.html#20205 
et al.


*** The website's still broken - I still can't mark data as "valid" or 
"invalid" even when logged in.


___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Christoph Hormann]

On Thursday 08 March 2018, Frederik Ramm wrote:
> [...]
>
> So even when we talk about desirable diversity goals, we have to
> remain practical; the goal of "a working OSM" is more important than
> the goal of "a diverse OSM" and diversity must take the back seat
> when it would make working in the project impossible.

I understand your point but i have to strongly disagree here.  IMO the 
English language dominance is one of the biggest structural problems of 
OSM in total and the biggest long term obstacle in becoming what OSM 
aims to be - the best map/geodatabase of the world.  And i completely 
agree this is a very hard problem, maybe even impossible to solve.  But 
that does not mean we should not try.

I don't want to appear fundamentalist on this - moving forward in small 
steps here is OK.  But giving up on meaningful language diversity in 
community communication because it is considered impractical while 
proudly communicating we want to support diversity otherwise would in 
my eyes be very hypocritical.

Also need to keep in mind that language diversity is strongly tied to 
cultural diversity because language always transports cultural values 
and conventions.

> We generally request that imports and automated edits are discussed
> before they are executed. The main reason for this is that we want to
> have a chance to discover flaws in the process, the licensing, or
> clean up misunderstandings. The import guidelines also say that
> "community buy-in" should be sought. Now if someone ran an import in
> Panama, then it would be a good idea to discuss this with the Panama
> community, and out of courtesy do in in Spanish. Ideally, the
> importer would be from Panama.

In the past it has been common practice to discuss imports on the local 
channels and this list.  Importers posting here without posting on the 
local channels first were usually asked to discuss it locally first - 
with the exception of cases were there clearly was no organized local 
community.  I also think there were imports across several countries in 
the past where several local communities have been consulted though 
probably never with a similar scope as here.

I am perfectly fine with discussing different modalities for 
multi-national imports but this should be a somewhat broader 
discussion.  If such a change in policy is established i am pretty sure 
there would be quickly others who want to do the same.  Facebook for 
example had already with their roads import indicated that they would 
prefer to move on to additional countries after they have finished with 
Thailand without the need to newly discuss their plans (and no, i don't 
want to compare Ilya's work here with that of Facebook, that would be 
ludicrous).

> But I can feel Ilya's exasperation at the suggestion of discussing a
> world-wide import on every local mailing list / forum / facebook
> group, in the language appropriate for each. This is not practical,
> and would kill the import in its tracks if we were to demand that.

I am not sure if i agree with that but it would probably be good if we 
had a list of feature counts by country, maybe separated into new 
features and features to be modified to put the discussion on a more 
solid level.

-- 
Christoph Hormann
http://www.imagico.de/

___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Clifford Snow]

On Thu, Mar 8, 2018 at 8:51 AM, Andy Townsend  wrote:

> On 07/03/2018 17:07, Ilya Zverev wrote:
>
>> You can see the numbers and check a few stations yourself at the
>> validation website (it will be very slow, please bear with it):
>>
>> http://audit.osmz.ru/project/navads_fuel
>>
>>
> Unfortunately it's not possible to provide any useful feedback at the
> moment because the website is "just sitting there".  Is there any way that
> the data could be presented so that people can actually view it, perhaps on
> a per-country basis?
>

Andy,
Ilya did provide a geojson [1] of the entire proposed import. If you look
back in the thread, IIya replied to my request. While large, it opens in
QGIS nicely.

[1]  https://transfer.sh/12oTYj/fuel.json


Clifford

-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Frederik Ramm]

Hi,

   the following might be a bit confusing to the casual reader because
I'm arguing in favour of Ilya's import even though I'd really prefer us
to have less imports, not more. But I think Ilya's proposal is being
criticised for the wrong reasons.

On 08.03.2018 17:07, Martin Koppenhoefer wrote:
> 2018-03-08 15:03 GMT+01:00 Ilya Zverev  >:
> 
>  Import Guidelines points to make me do tons of extra community
> work, like finding communication channels for ~20 countries and
> convincing every single mapper on these, in their local languages.
> 
> maybe we can improve the documentation for the primary comunity
> comunication channels per country, on a single (hopefully up to date) page?
> 
> I think it is generally a good provision of the guidelines to ask the
> local community for comments.

I think there's certainly a practical limit to diversity, especially for
a small project/organisation like ours.

If you don't speak English, then there are areas of OpenStreetMap where
it is difficult for you to make your voice heard. You cannot join the
OSMF board if you don't speak English; and even for passing a meaningful
vote on who should be on the board, you will have to rely on trusted
third parties to explain to you what the candidates stand for. You will
have difficulty participating in most of the international working
groups, in tagging discussions, or international OSM software
development. That's a fact and it would require a very large number of
volunteers or a very big amount of money to change that. Requesting that
everything that could possibly affect the OSM community in a country is
also accessible to them when they don't speak English might sound great
but it would definitely make most of the project grind to a halt.

So even when we talk about desirable diversity goals, we have to remain
practical; the goal of "a working OSM" is more important than the goal
of "a diverse OSM" and diversity must take the back seat when it would
make working in the project impossible.

We generally request that imports and automated edits are discussed
before they are executed. The main reason for this is that we want to
have a chance to discover flaws in the process, the licensing, or clean
up misunderstandings. The import guidelines also say that "community
buy-in" should be sought. Now if someone ran an import in Panama, then
it would be a good idea to discuss this with the Panama community, and
out of courtesy do in in Spanish. Ideally, the importer would be from
Panama.

But I can feel Ilya's exasperation at the suggestion of discussing a
world-wide import on every local mailing list / forum / facebook group,
in the language appropriate for each. This is not practical, and would
kill the import in its tracks if we were to demand that.

Five or ten years in the future, when we have established local chapters
around the globe, maybe then we'll have a mechanism to submit an import
proposal and then have each country say yes or no after consulting with
their folks in their language, but we're not there yet.

My practical suggestion would be to:

* discuss the import here, in English;
* make a list of countries most affected by the import (perhaps all with
more than 1000 edits - choose a practical threshold) and if they have a
talk-xx mailing list, make a posting there ("I am planning to import
1000 fuel stations in your country, discussion over on imports@")
* if there's significant opposition overall, scrap the import altogether
* else, split up the import by country, and run the import in all
countries potentially excluding those where people have objected
* if, after the import, voices pop up complaining because they were
unaware of the discussion and the import somehow breaks something in
their country - simply revert for that particular country

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Andy Townsend]

On 07/03/2018 17:07, Ilya Zverev wrote:

You can see the numbers and check a few stations yourself at the validation 
website (it will be very slow, please bear with it):

http://audit.osmz.ru/project/navads_fuel



Unfortunately it's not possible to provide any useful feedback at the 
moment because the website is "just sitting there".  Is there any way 
that the data could be presented so that people can actually view it, 
perhaps on a per-country basis?


Best Regards,

Andy


___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Jason Remillard]

Hi,

I checked 20 or so stations in Massachusetts, they all look good.

Jason

On Wed, Mar 7, 2018 at 12:07 PM, Ilya Zverev  wrote:

> Hi everyone,
>
> Following the recent UK Shell stations import, I've got ahold of the
> entire NavAds dataset. A major part of it are fuel stations all across the
> world: UK, US, France, Germany, Australia, and many other countries. We
> have the full permission to use the data, it is very recent (we get updates
> monthly) and quite precise: all locations are manually tweaked after
> geocoding. It has fuel stations of major brands (Shell, BP, Total) and
> smaller, non-chain owners (ProxiFuel, Lotherol, Edeka).
>
> You can see the numbers and check a few stations yourself at the
> validation website (it will be very slow, please bear with it):
>
> http://audit.osmz.ru/project/navads_fuel
>
> I want to import this data to OpenStreetMap. Which means, to update tags
> on 38k fuel stations and add 21k stations not present on OSM. For that, I
> need you to have a look at the data. Of course it is impossible to manually
> check every object, and to do that afterwards each month up to eternity.
> But you can help check that tags are correct, that there are no duplicates
> being created, and that postcodes on objects are good to import.
>
> The source dataset has much more information than I upload. For example,
> it has flags for grocery stores, car washes, restaurants in fuel station
> buildings, and I don't know tags for these. It has street addresses, but
> they need to be parsed, and that isn't easy ("62 Rue Du General Leclerc",
> "Pritzwalker Straße 20", "Allée de Provence Bld Des Alpes De Haute
> Provence"). It has town and state and country for each fuel station. You
> can see a sample of the source json here, and maybe suggest on using some
> of the extra information:
>
> https://pastebin.com/SLXq8Eb8
>
> Please be gentle. I know some of you despise large-scale imports, but this
> one to me is as good as imports get. I'd like to get it to completion, even
> if all that remains from the source data are the type and the brand name.
>
> Thanks,
> Ilya
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Martin Koppenhoefer]

2018-03-08 15:03 GMT+01:00 Ilya Zverev :

>  Import Guidelines points to make me do tons of extra community work,
> like finding communication channels for ~20 countries and convincing every
> single mapper on these, in their local languages.



maybe we can improve the documentation for the primary comunity
comunication channels per country, on a single (hopefully up to date) page?

I think it is generally a good provision of the guidelines to ask the local
community for comments.

Cheers,
Martin
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Blake Girardot]

On Thu, Mar 8, 2018 at 4:38 PM, Levente Juhász  wrote:
> Thanks for the clarifications, Ilya. I started typing my response as I read
> your original mail and gave my first impressions. Some questions got
> answered after checking the data and reading through your entire mail but
> remained in my response. Anyway, 2nd round below:
>
> As for 0), I was thinking two things. You're right about adding new,
> existing stations not being overrides, there's no question about that.
> However, there's still the questions of the overall currentness of the
> NavAds data. For example, to get an idea about how long it takes on average
> to remove a permanently closed gas station from the dataset. If it takes too
> long, then there might be a monthly add-remove-add battle between you and a
> local mapper, which is not ideal. I later checked my area and stations
> looked good (i.e. no closed gas stations in the NavAds data). Hopefully as
> more people review the data we'll see that this is not an issue.

Just to be clear, I fully support this import 100%, especially since
Ilya is involved. Nothing is perfect, even craft mapping, so I expect
that will be the case with any import.

To your question and maybe this offers some insight, it is just one data point:

http://audit.osmz.ru/browse/navads_fuel/NVDS12_3416

That is on the wrong building so misplaced about 50m (it should have
been on the building just to the SE)

It has also been torn down never to be replaced about 1 - 2 months ago
so should not be in the dataset.

I hope that helps in some small way.

Thank you for your work on this Ilya!

Cheers
blake


Blake Girardot
OSM Wiki - https://wiki.openstreetmap.org/wiki/User:Bgirardot
HOTOSM Member - https://hotosm.org/users/blake_girardot
skype: jblakegirardot

___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Levente Juhász]

Thanks for the clarifications, Ilya. I started typing my response as I read
your original mail and gave my first impressions. Some questions got
answered after checking the data and reading through your entire mail but
remained in my response. Anyway, 2nd round below:

As for 0), I was thinking two things. You're right about adding new,
existing stations not being overrides, there's no question about that.
However, there's still the questions of the overall currentness of the
NavAds data. For example, to get an idea about how long it takes on average
to remove a permanently closed gas station from the dataset. If it takes
too long, then there might be a monthly add-remove-add battle between you
and a local mapper, which is not ideal. I later checked my area and
stations looked good (i.e. no closed gas stations in the NavAds data).
Hopefully as more people review the data we'll see that this is not an
issue. Another thing I had in mind was about tags. For example, I wasn't
sure what tags you're planning to update and how. Like updating operator to
"Shell Oil Company" from "Shell" didn't seem like a good idea if a local
community already agreed on using "Shell". Same for other tags. Now knowing
that this is in fact not part of your import, I don't have any problems
with it. It was more like a hypothetical question since the whole process
wasn't entirely clear (which again comes down to having a documentation).

1) Fair enough. I wasn't aware of the different notations for different
countries. It just looked strange at first but now it makes sense.

2) Great.

3) OK, hopefully it's not a common case.

As for easing the tension about cross-country imports, you could for
example divide the import changeset into multiple changesets for each
country. With that, even if some concerns or a strong opposition arise from
a local community, it would only affect one changeset out of the ~20 total.
Since you have addresses this could be done without spatial operations.
It's just something to think about.

Cheers,
Levente


On Thu, Mar 8, 2018 at 9:17 AM Ilya Zverev  wrote:

> Hi Levente,
>
> Thanks for reviewing the import. You have many valid points, which
> mostly come from the lack of documentation on my side.
>
> 0) Why do you think this import overrides any work? Adding new fuel
> stations certainly does not, so does adding phones and websites hurt the
> mappers? I'd be glad and more understanding of local mappers if you
> elaborated on this.
>
> 1) The wiki page you link to explicitly tells to use the E.164 standard,
> which mandates different notation for different countries. The wiki
> specifically mentions the format for US phone numbers. For formatting
> the numbers, I am using the Google's libphonenumber library. It is the
> same that is used by Android phones to format numbers, and I haven't
> seen any critique on these phones' formatting errors.
>
> 2) I guess these links were pointing to specific stations, but the Shell
> website was changed a while ago, making these links invalid. I'll
> contact NavAds about that.
>
> 3) With that gas station you have found that the source dataset has
> duplicates. OSM Conflator has not means of detecting that. But I will
> definitely look for any other duplicates, remove them and report back to
> NavAds.
>
> Thanks,
> Ilya
>
> 07.03.2018 22:32, Levente Juhász пишет:
> > Hi All,
> >
> > Generally, I think imports can be super useful if carefully executed.
> > This includes precise documentation. More importantly, as
> > Christoph pointed out, local communities should be involved as we just
> > simply don't know whether they've already put efforts into mapping gas
> > stations, or if they've already agreed on some country specific best
> > practices. I don't think overriding their previous work with a global
> > import is a good idea.
> >
> > Also, I checked a few data points manually and have the following
> comments:
> >
> > 1) Phone number patterns should follow the same rules within the
> > dataset. E.g. "+36 62 464 024 <+36%2062%20464%20024>"
> > (http://audit.osmz.ru/browse/navads_fuel/NVDS353_10201112) in Hungary vs
> > "+1 561-544-6012 <(561)%20544-6012>"
> > (http://audit.osmz.ru/browse/navads_fuel/NVDS353_10008561) in the US
> > (dashes/no dashes in the local part of the number). See the wiki
> > (https://wiki.openstreetmap.org/wiki/Key:phone#Usage) for example
> patterns.
> >
> > 2) Some urls don't point to the specific station. E.g. for this station:
> > http://audit.osmz.ru/browse/navads_fuel/NVDS353_10201112
> >   website=
> http://www.shell.hu/products-services/on-the-road/shell-station-locator.html?id=10201112=true
>  will
> be redirected to:
> https://www.shell.hu/autosok/shell-station-locator.html#vanity-aHR0cHM6Ly93d3cuc2hlbGwuaHUvcHJvZHVjdHMtc2VydmljZXMvb24tdGhlLXJvYWQvc2hlbGwtc3RhdGlvbi1sb2NhdG9yLmh0bWw
> which is not a unique page of that station. Same thing with Shell in Poland
> (e.g. 

Re: [Imports] Worldwide fuel stations import, 59k objects

[Tommy Bruce]

To be fair I think it is important that importa are a little difficult. The 
review process should take some legwork. And steering new mappers away from 
imports is a good idea. Imports can really screw up the map when they are done 
wrong. Making mappers create wiki pages and contact other mappers first makes 
sure they are experienced with OSM and the community, but also gives time for 
people to point out problems. A perfect example would be importing parcel 
boundaries. A new mappers might look at the map and think hey we should import 
parcel boundaries and label them boundary level 10 or something. Any 
experienced mapper knows that the OSM community has decided against importing 
parcel boundaries. But if they missed that along the way it would hopefully be 
caught in the review process where people could point out OSM does not do 
parcel boundaries. Contacting mappers in other countries in other languages in 
my opinion should be the easiest part of the import. 

You can use http://resultmaps.neis-one.org/oooc to identify who is active in 
the areas and contact them on open street map. Google translate is your friend. 
But also English is the main language online including for OSM tags so chances 
are they probably can figure out what you are trying to say if you write it in 
English. 

Sent from my iPhone

> On Mar 8, 2018, at 08:11, Ilya Zverev  wrote:
> 
> 08.03.2018 10:28, Frederik Ramm пишет:
>> Ilya,
>> as imports go, yours tend to be the best-planned and I appreciate that
>> you do respect our processes even in the face of adversity. I would
>> encourage you to also make them the best-documented ;)
> 
> Thanks Frederik, I'll try making a good wiki page for these. Though I'm 
> inclined to make a single page for every POI import done with OSM Conflator, 
> since only tags and dataset providers differ for these, and algorithms are 
> mostly the same.
> 
>> You say that "we get updates monthly" and later that it is "impossible
>> to check each object ... afterwards each month up to eternity". Does
>> that mean that you are not planning a one-off import, but a continuous
>> data synchronisation process between the NavAds data set and OSM? If
>> that is the case, it should be said clearly and mandates a closer look.
> 
> Yes, I am receving a json file with ~300k features every month from NavAds. 
> This import was made from the March dataset, I also have one from February. 
> After a successful first import, I plan to update the data with new 
> information. That's why the import sets "ref:navads" tags, for easier future 
> conflation.
> 
>> I would like to understand more about the different parties in this game
>> and their interests. I assume that the fuel station chains pay someone
>> (Navads?) to publicize their information, and Navads in turn pays
>> someone (you? your employer?) to get this information into OSM. I'm not
>> sure, you say "I got my hands on this data set" which sounds a bit like
>> you're doing this in a hobby capacity but later you say "we".
> 
> Last year, MAPS.ME has signed some kind of a agreement with NavAds and some 
> other SEO companies, stating that we will help them with getting their data 
> on OpenStreetMap. You have seen the Shell stations import and a couple 
> imports by Brandify (Walmarts and Retro Fitness gyms) in US, that were the 
> result of this collaboration.
> 
> No money switched hands here. We — as in MAPS.ME — are doing this because it 
> makes the map that we use richer and more useful to people, especially 
> tourists. SEO companies do that because they are paid for it by companies.
> 
> Finally, you could say I am doing this as a hobby. Because this is not 
> directly related to the improvement of the application, nor this in any way 
> advertises MAPS.ME. It distracts me from work I need to be doing, that have 
> deadlines. I just said that I could help in the past, and I am spending my 
> time on this because I myself believe OSM should have the most up-to-date 
> information on POIs. Craftmapping helps with that, but mappers are not 
> everywhere, and most doesn't bother themselves with stopping at every fuel 
> station and noting their opening hours.
> 
>> <...> Paying brands will be
>> fully covered even in areas with no mapping activity, smaller
>> independent stations and non-paying brands will only be covered in areas
>> where a mapper happened to add them >
>> Now you may reply "but nobody is keeping anyone from adding a small
>> independent fuel station" and you are right; but in letting the major
>> brands directly manage their representation on OSM you are getting one
>> step closer to making OSM unwelcoming for independent contributions
>> ("it's no use adding this single shop here, all the retail data in OSM
>> is managed by the big brands anyway").
> 
> I understand what you are getting at. I am also bothered that not all the 
> fuel station owners know about companies like NavAds and use their services. 
> That 

Re: [Imports] Worldwide fuel stations import, 59k objects

[Christoph Hormann]

On Thursday 08 March 2018, Ilya Zverev wrote:
> It hurts me a bit that instead of clearly and unambigously saying "I
> am against this and any other cross-country imports, regardless of
> their nature and sources", you are pulling all the obscure Import
> Guidelines points to make me do tons of extra community work, like
> finding communication channels for ~20 countries and convincing every
> single mapper on these, in their local languages.
>
> This is what people are talking when they write about CoC. [...]

I am not sure if you are aware of the irony here - you criticize me for 
citing and arguing with codified standards and in the next sentence 
argue with the need for codified standards to prevent me arguing by 
citing codified standards...

My aim here was to participate in an import review under the import 
guidelines and as part of this i considered it appropriate to refer to 
these guidelines and what seems to be missing in this and the previous 
import you refer to w.r.t. those.

If that was not your intention with your mail i apologize for 
misunderstanding you.  If you want to in general discuss the 
possibility of worldwide imports and use the fuel stations merely as an 
example that is a different matter and criticizing you for not yet 
having proper documentation for this was premature.

I would however in that case re-affirm my assessment that the local 
communities affected need to have the opportunity to review and discuss 
and possibly reject or request changes for the import plan for their 
respective country and not just collectively world wide.  The primacy 
of the local community is a a very fundamental principle of OSM in my 
eyes and it is outside the scope of this mailing list to supersede it.  

One way this could be accomplished efficiently is by recruiting 
rapporteurs for your plan for the different local communities who 
manage the local consultation.  If the plan is seen with favour in 
general it should not be too difficult to find such people.

You could also consider excluding smaller countries with just a handful 
of features from the import plan and manually review and add the data 
there (again possibly with the help of interested local mappers) 
without an import.

And since you seem to make a different assumption - for my own local 
community in Germany i think this data is probably quite useful - not 
necessarily for an import (there would likely be very few features to 
be newly added anyway so it would be more a mechanical edit anyway) but 
maybe more for checking completeness and finding missing features (in a 
similar way as we do with address lists to check completeness of 
addresses).

-- 
Christoph Hormann
http://www.imagico.de/

___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Ilya Zverev]

Hi Levente,

Thanks for reviewing the import. You have many valid points, which 
mostly come from the lack of documentation on my side.


0) Why do you think this import overrides any work? Adding new fuel 
stations certainly does not, so does adding phones and websites hurt the 
mappers? I'd be glad and more understanding of local mappers if you 
elaborated on this.


1) The wiki page you link to explicitly tells to use the E.164 standard, 
which mandates different notation for different countries. The wiki 
specifically mentions the format for US phone numbers. For formatting 
the numbers, I am using the Google's libphonenumber library. It is the 
same that is used by Android phones to format numbers, and I haven't 
seen any critique on these phones' formatting errors.


2) I guess these links were pointing to specific stations, but the Shell 
website was changed a while ago, making these links invalid. I'll 
contact NavAds about that.


3) With that gas station you have found that the source dataset has 
duplicates. OSM Conflator has not means of detecting that. But I will 
definitely look for any other duplicates, remove them and report back to 
NavAds.


Thanks,
Ilya

07.03.2018 22:32, Levente Juhász пишет:

Hi All,

Generally, I think imports can be super useful if carefully executed. 
This includes precise documentation. More importantly, as 
Christoph pointed out, local communities should be involved as we just 
simply don't know whether they've already put efforts into mapping gas 
stations, or if they've already agreed on some country specific best 
practices. I don't think overriding their previous work with a global 
import is a good idea.


Also, I checked a few data points manually and have the following comments:

1) Phone number patterns should follow the same rules within the 
dataset. E.g. "+36 62 464 024" 
(http://audit.osmz.ru/browse/navads_fuel/NVDS353_10201112) in Hungary vs 
"+1 561-544-6012" 
(http://audit.osmz.ru/browse/navads_fuel/NVDS353_10008561) in the US 
(dashes/no dashes in the local part of the number). See the wiki 
(https://wiki.openstreetmap.org/wiki/Key:phone#Usage) for example patterns.


2) Some urls don't point to the specific station. E.g. for this station: 
http://audit.osmz.ru/browse/navads_fuel/NVDS353_10201112  
  website=http://www.shell.hu/products-services/on-the-road/shell-station-locator.html?id=10201112=true will be redirected to: https://www.shell.hu/autosok/shell-station-locator.html#vanity-aHR0cHM6Ly93d3cuc2hlbGwuaHUvcHJvZHVjdHMtc2VydmljZXMvb24tdGhlLXJvYWQvc2hlbGwtc3RhdGlvbi1sb2NhdG9yLmh0bWw  which is not a unique page of that station. Same thing with Shell in Poland (e.g. http://audit.osmz.ru/browse/navads_fuel/NVDS353_10034854). In these cases, a general website pointing to www.shell.hu  or www.shell.pl  would be a better choice if you want to add a website. Additional url parameters here just don't serve any purpose without the correct pattern, therefore I don't think they should be used added.


I'm not familiar with the OSM Conflator tool but it would be great to 
know what parameters it uses for finding already existing OSM features. 
I randomly found the following example: 
http://audit.osmz.ru/browse/navads_fuel/NVDS106_1073560996PL0 which 
shows a newly created feature (green) and a feature to be modified 
(blue). In this case, those feature refer to the same gas station so it 
should be a simple update. I'm wondering about the conflation parameters 
and if you've tested different values and evaluated the differences. I'm 
also wondering if you have a general idea about the number of similar 
cases. This is the information that would be helpful in the import 
documentation. This scenario is also related to 1) above since the phone 
number is about to be updated with the same value, in a different 
format. Quite possibly the updated pattern is the correct one, but I'd 
ask the Polish community whether the old value was intentional or not.


All in all, this could be a great addition, but I think the import 
workflow needs some more work.


Cheers,
Levente

On Wed, Mar 7, 2018 at 1:09 PM Christoph Hormann > wrote:


On Wednesday 07 March 2018, Ilya Zverev wrote:
 > Hi everyone,
 >
 > Following the recent UK Shell stations import, I've got ahold of the
 > entire NavAds dataset. A major part of it are fuel stations all
 > across the world: UK, US, France, Germany, Australia, and many other
 > countries. [...]

I don't want to comment on the import itself - have done so in the past,
nothing really to add - except maybe that i looked for documentation of
the mentioned UK Shell stations import on the wiki or an entry in the
import catalogue - both of which are required by the import guidelines
and neither of which seems to exist (and neither for this import
apparently).

The import guidelines also clearly state 

Re: [Imports] Worldwide fuel stations import, 59k objects

[Ilya Zverev]

It hurts me a bit that instead of clearly and unambigously saying "I am 
against this and any other cross-country imports, regardless of their 
nature and sources", you are pulling all the obscure Import Guidelines 
points to make me do tons of extra community work, like finding 
communication channels for ~20 countries and convincing every single 
mapper on these, in their local languages.


This is what people are talking when they write about CoC. I imagine you 
think your message is clear, to the point, and "business as usual". But 
if I were new here, I'd slam the door muttering "to hell with OSM, never 
again".


The only constructive suggestion I could dig from your message (again, 
deeply buried in sarcasm) is that I should make a wiki page. I did it 
for all the past POI imports, and will do one for this import. Sorry 
that I forgot this time.


I am certainly looking forward to another article of yours on the 
negative impact of large POI imports. I did many conference talks on the 
positive impact already, and only Frederik has articulated his 
alternative opinion clearly.


Or, if you want to be more positive, could you outline an acceptable 
workflow for such imports, that do not require learning dozen of foreign 
languages and going through a hundred mappers individually.


Ilya

07.03.2018 21:07, Christoph Hormann пишет:

On Wednesday 07 March 2018, Ilya Zverev wrote:

Hi everyone,

Following the recent UK Shell stations import, I've got ahold of the
entire NavAds dataset. A major part of it are fuel stations all
across the world: UK, US, France, Germany, Australia, and many other
countries. [...]


I don't want to comment on the import itself - have done so in the past,
nothing really to add - except maybe that i looked for documentation of
the mentioned UK Shell stations import on the wiki or an entry in the
import catalogue - both of which are required by the import guidelines
and neither of which seems to exist (and neither for this import
apparently).

The import guidelines also clearly state that

"You must not import the data without local buy-in"

which leads me to conclude that for a multi-country import you have to
consult with each of the local communities affected individually.

The local communities need to have the right to object to the import or
to have specific local conventions regarding tagging that the import
needs to follow in their domain.  Local mappers must not be required to
write here in English to ask questions and raise concens about local
aspects to be heard.




___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Mike N]

On 3/8/2018 2:28 AM, Frederik Ramm wrote:

Paying brands will be
fully covered even in areas with no mapping activity, smaller
independent stations and non-paying brands will only be covered in areas
where a mapper happened to add them.


In my US state at least, the import will add no new stations, but adds / 
clarifies information on about 4 existing stations.   I'm suspecting 
this applies generally to the US where addresses have not been entered 
in OSM and there is no open geocoder.


___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Frederik Ramm]

Ilya,

as imports go, yours tend to be the best-planned and I appreciate that
you do respect our processes even in the face of adversity. I would
encourage you to also make them the best-documented ;)

You say that "we get updates monthly" and later that it is "impossible
to check each object ... afterwards each month up to eternity". Does
that mean that you are not planning a one-off import, but a continuous
data synchronisation process between the NavAds data set and OSM? If
that is the case, it should be said clearly and mandates a closer look.

I would like to understand more about the different parties in this game
and their interests. I assume that the fuel station chains pay someone
(Navads?) to publicize their information, and Navads in turn pays
someone (you? your employer?) to get this information into OSM. I'm not
sure, you say "I got my hands on this data set" which sounds a bit like
you're doing this in a hobby capacity but later you say "we". Anyhow,
this means that what we have here in the end is businesses paying money
to appear on OSM. I don't assume that anyone in this chain - the fuel
station chains, Navads, you, your employer - has any interest of
enriching the data set with information about those fuel stations that
don't pay. This leads to a skewed picture on OSM: Paying brands will be
fully covered even in areas with no mapping activity, smaller
independent stations and non-paying brands will only be covered in areas
where a mapper happened to add them.

This is not a blocker, but I would like us all to think about this and
be aware of it; in allowing imports like this to happen, our content
gets skewed towards business interests.

Now you may reply "but nobody is keeping anyone from adding a small
independent fuel station" and you are right; but in letting the major
brands directly manage their representation on OSM you are getting one
step closer to making OSM unwelcoming for independent contributions
("it's no use adding this single shop here, all the retail data in OSM
is managed by the big brands anyway").

I am also concerned about diversity. In other places we make big words
about how it is important to attract people from all walks of life, all
nationalities, all genders to OSM because we believe that this will make
the map better. But those words ring hollow if at the same time, with an
import like this, we're essentially replacing the voice of several
thousand mappers who have last edited a fuel station, by the voice of
one commercial data provider because we believe that their data is
somehow better. What is diversity for, then, when in the end we will
always assume that commercial data is better? Is diversity just for
mapping the parts of OSM that are of no interest to business, and for
everything else we won't hesitate to let our imperfect data made by all
these imperfect people from all walks of life, be overridden by the
shiny corporate data set?

Again, not something that would block this particular import, but
something to keep in mind. This import is one small step on the road
towards an explicitly non-diverse, corporate-managed OSM and we have to
be very careful about just how far we're prepared to go down that road
in return for nice data. Because after the fuel stations there will come
the restaurant chains, and then the hotel data sets, the retail chains,
and we might well look back at this import in ten year's time and say:
This is where the gentrification began in OSM.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Clifford Snow]

I did a check in my area in western Washington State - they were all okay.
One new stations (more on that later) but some additional information on
others. The existing stations used existing nodes so spatial accuracy is
the same as what is there today. The additional information available might
be worth adding. For addresses that might mean having to process by country
or countries with similar addressing schemes.

In my area, the data included Shell and ARCO stations. Unfortunately
neither website provides and structured data (schema.org) that could be
used to improve the richness of the data. ARCO website didn't even list any
stations in my area, even though there are many.

The one new station actually is in OSM, but only with a name and building
tag. The spatial accuracy was very good.

I also found three stations to be deleted:
https://www.openstreetmap.org/node/5295584324
https://www.openstreetmap.org/node/243343385
https://www.openstreetmap.org/node/249400366

I would say go ahead with an import but ideally see if address information
could be added.

Clifford

On Wed, Mar 7, 2018 at 9:07 AM, Ilya Zverev  wrote:

> Hi everyone,
>
> Following the recent UK Shell stations import, I've got ahold of the
> entire NavAds dataset. A major part of it are fuel stations all across the
> world: UK, US, France, Germany, Australia, and many other countries. We
> have the full permission to use the data, it is very recent (we get updates
> monthly) and quite precise: all locations are manually tweaked after
> geocoding. It has fuel stations of major brands (Shell, BP, Total) and
> smaller, non-chain owners (ProxiFuel, Lotherol, Edeka).
>
> You can see the numbers and check a few stations yourself at the
> validation website (it will be very slow, please bear with it):
>
> http://audit.osmz.ru/project/navads_fuel
>
> I want to import this data to OpenStreetMap. Which means, to update tags
> on 38k fuel stations and add 21k stations not present on OSM. For that, I
> need you to have a look at the data. Of course it is impossible to manually
> check every object, and to do that afterwards each month up to eternity.
> But you can help check that tags are correct, that there are no duplicates
> being created, and that postcodes on objects are good to import.
>
> The source dataset has much more information than I upload. For example,
> it has flags for grocery stores, car washes, restaurants in fuel station
> buildings, and I don't know tags for these. It has street addresses, but
> they need to be parsed, and that isn't easy ("62 Rue Du General Leclerc",
> "Pritzwalker Straße 20", "Allée de Provence Bld Des Alpes De Haute
> Provence"). It has town and state and country for each fuel station. You
> can see a sample of the source json here, and maybe suggest on using some
> of the extra information:
>
> https://pastebin.com/SLXq8Eb8
>
> Please be gentle. I know some of you despise large-scale imports, but this
> one to me is as good as imports get. I'd like to get it to completion, even
> if all that remains from the source data are the type and the brand name.
>
> Thanks,
> Ilya
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>



-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Alan Richards]

Here in Canada there was a push recently to add hundreds or thousands of
missing gas stations across the country recently.
It seems this dataset includes the Shell stations only (at least in BC),
but it does seem generally accurate and helpful. The positions match very
closely with existing stations and satellite imagery, and even for existing
stations it adds postcode, opening hours, phone numbers.

My concern would be whether all these stations are correct today. I did
notice one station in the map that closed in 2014/2015 and was removed from
OSM, yet appears in this dataset. We don't want to be re-adding long-closed
stations.

On Wed, Mar 7, 2018 at 10:07 AM, Christoph Hormann  wrote:

> On Wednesday 07 March 2018, Ilya Zverev wrote:
> > Hi everyone,
> >
> > Following the recent UK Shell stations import, I've got ahold of the
> > entire NavAds dataset. A major part of it are fuel stations all
> > across the world: UK, US, France, Germany, Australia, and many other
> > countries. [...]
>
> I don't want to comment on the import itself - have done so in the past,
> nothing really to add - except maybe that i looked for documentation of
> the mentioned UK Shell stations import on the wiki or an entry in the
> import catalogue - both of which are required by the import guidelines
> and neither of which seems to exist (and neither for this import
> apparently).
>
> The import guidelines also clearly state that
>
> "You must not import the data without local buy-in"
>
> which leads me to conclude that for a multi-country import you have to
> consult with each of the local communities affected individually.
>
> The local communities need to have the right to object to the import or
> to have specific local conventions regarding tagging that the import
> needs to follow in their domain.  Local mappers must not be required to
> write here in English to ask questions and raise concens about local
> aspects to be heard.
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Levente Juhász]

Hi All,

Generally, I think imports can be super useful if carefully executed. This
includes precise documentation. More importantly, as Christoph pointed out,
local communities should be involved as we just simply don't know whether
they've already put efforts into mapping gas stations, or if they've
already agreed on some country specific best practices. I don't think
overriding their previous work with a global import is a good idea.

Also, I checked a few data points manually and have the following comments:

1) Phone number patterns should follow the same rules within the dataset.
E.g. "+36 62 464 024" (
http://audit.osmz.ru/browse/navads_fuel/NVDS353_10201112) in Hungary vs "+1
561-544-6012" (http://audit.osmz.ru/browse/navads_fuel/NVDS353_10008561) in
the US (dashes/no dashes in the local part of the number). See the wiki (
https://wiki.openstreetmap.org/wiki/Key:phone#Usage) for example patterns.

2) Some urls don't point to the specific station. E.g. for this station:
http://audit.osmz.ru/browse/navads_fuel/NVDS353_10201112   website=
http://www.shell.hu/products-services/on-the-road/shell-station-locator.html?id=10201112=true
will
be redirected to:
https://www.shell.hu/autosok/shell-station-locator.html#vanity-aHR0cHM6Ly93d3cuc2hlbGwuaHUvcHJvZHVjdHMtc2VydmljZXMvb24tdGhlLXJvYWQvc2hlbGwtc3RhdGlvbi1sb2NhdG9yLmh0bWw
which is not a unique page of that station. Same thing with Shell in Poland
(e.g. http://audit.osmz.ru/browse/navads_fuel/NVDS353_10034854). In these
cases, a general website pointing to www.shell.hu or www.shell.pl would be
a better choice if you want to add a website. Additional url parameters
here just don't serve any purpose without the correct pattern, therefore I
don't think they should be used added.

I'm not familiar with the OSM Conflator tool but it would be great to know
what parameters it uses for finding already existing OSM features. I
randomly found the following example:
http://audit.osmz.ru/browse/navads_fuel/NVDS106_1073560996PL0 which shows a
newly created feature (green) and a feature to be modified (blue). In this
case, those feature refer to the same gas station so it should be a simple
update. I'm wondering about the conflation parameters and if you've tested
different values and evaluated the differences. I'm also wondering if you
have a general idea about the number of similar cases. This is the
information that would be helpful in the import documentation. This
scenario is also related to 1) above since the phone number is about to be
updated with the same value, in a different format. Quite possibly the
updated pattern is the correct one, but I'd ask the Polish community
whether the old value was intentional or not.

All in all, this could be a great addition, but I think the import workflow
needs some more work.

Cheers,
Levente

On Wed, Mar 7, 2018 at 1:09 PM Christoph Hormann  wrote:

> On Wednesday 07 March 2018, Ilya Zverev wrote:
> > Hi everyone,
> >
> > Following the recent UK Shell stations import, I've got ahold of the
> > entire NavAds dataset. A major part of it are fuel stations all
> > across the world: UK, US, France, Germany, Australia, and many other
> > countries. [...]
>
> I don't want to comment on the import itself - have done so in the past,
> nothing really to add - except maybe that i looked for documentation of
> the mentioned UK Shell stations import on the wiki or an entry in the
> import catalogue - both of which are required by the import guidelines
> and neither of which seems to exist (and neither for this import
> apparently).
>
> The import guidelines also clearly state that
>
> "You must not import the data without local buy-in"
>
> which leads me to conclude that for a multi-country import you have to
> consult with each of the local communities affected individually.
>
> The local communities need to have the right to object to the import or
> to have specific local conventions regarding tagging that the import
> needs to follow in their domain.  Local mappers must not be required to
> write here in English to ask questions and raise concens about local
> aspects to be heard.
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Christoph Hormann]

On Wednesday 07 March 2018, Ilya Zverev wrote:
> Hi everyone,
>
> Following the recent UK Shell stations import, I've got ahold of the
> entire NavAds dataset. A major part of it are fuel stations all
> across the world: UK, US, France, Germany, Australia, and many other
> countries. [...]

I don't want to comment on the import itself - have done so in the past, 
nothing really to add - except maybe that i looked for documentation of 
the mentioned UK Shell stations import on the wiki or an entry in the 
import catalogue - both of which are required by the import guidelines 
and neither of which seems to exist (and neither for this import 
apparently).

The import guidelines also clearly state that

"You must not import the data without local buy-in"

which leads me to conclude that for a multi-country import you have to 
consult with each of the local communities affected individually.

The local communities need to have the right to object to the import or 
to have specific local conventions regarding tagging that the import 
needs to follow in their domain.  Local mappers must not be required to 
write here in English to ask questions and raise concens about local 
aspects to be heard.

-- 
Christoph Hormann
http://www.imagico.de/

___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Ilya Zverev]

Here is the GeoJSON file for the import (42 MB — don't try to open it in 
geojson.io, use QGIS): https://transfer.sh/12oTYj/fuel.json

Ilya

> 7 марта 2018 г., в 20:18, Clifford Snow  написал(а):
> 
> Ilya,
> Can you place the entire db online somewhere so we can look at a sample in 
> our area? I tried to find some stations near me but the browse as points 
> failed and looking at the data as a table isn't very user friendly.
> 
> Thanks,
> Clifford
> 
> On Wed, Mar 7, 2018 at 9:07 AM, Ilya Zverev  wrote:
> Hi everyone,
> 
> Following the recent UK Shell stations import, I've got ahold of the entire 
> NavAds dataset. A major part of it are fuel stations all across the world: 
> UK, US, France, Germany, Australia, and many other countries. We have the 
> full permission to use the data, it is very recent (we get updates monthly) 
> and quite precise: all locations are manually tweaked after geocoding. It has 
> fuel stations of major brands (Shell, BP, Total) and smaller, non-chain 
> owners (ProxiFuel, Lotherol, Edeka).
> 
> You can see the numbers and check a few stations yourself at the validation 
> website (it will be very slow, please bear with it):
> 
> http://audit.osmz.ru/project/navads_fuel
> 
> I want to import this data to OpenStreetMap. Which means, to update tags on 
> 38k fuel stations and add 21k stations not present on OSM. For that, I need 
> you to have a look at the data. Of course it is impossible to manually check 
> every object, and to do that afterwards each month up to eternity. But you 
> can help check that tags are correct, that there are no duplicates being 
> created, and that postcodes on objects are good to import.
> 
> The source dataset has much more information than I upload. For example, it 
> has flags for grocery stores, car washes, restaurants in fuel station 
> buildings, and I don't know tags for these. It has street addresses, but they 
> need to be parsed, and that isn't easy ("62 Rue Du General Leclerc", 
> "Pritzwalker Straße 20", "Allée de Provence Bld Des Alpes De Haute 
> Provence"). It has town and state and country for each fuel station. You can 
> see a sample of the source json here, and maybe suggest on using some of the 
> extra information:
> 
> https://pastebin.com/SLXq8Eb8
> 
> Please be gentle. I know some of you despise large-scale imports, but this 
> one to me is as good as imports get. I'd like to get it to completion, even 
> if all that remains from the source data are the type and the brand name.
> 
> Thanks,
> Ilya
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
> 
> 
> 
> -- 
> @osm_seattle
> osm_seattle.snowandsnow.us
> OpenStreetMap: Maps with a human touch


___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

Re: [Imports] Worldwide fuel stations import, 59k objects

[Clifford Snow]

Ilya,
Can you place the entire db online somewhere so we can look at a sample in
our area? I tried to find some stations near me but the browse as points
failed and looking at the data as a table isn't very user friendly.

Thanks,
Clifford

On Wed, Mar 7, 2018 at 9:07 AM, Ilya Zverev  wrote:

> Hi everyone,
>
> Following the recent UK Shell stations import, I've got ahold of the
> entire NavAds dataset. A major part of it are fuel stations all across the
> world: UK, US, France, Germany, Australia, and many other countries. We
> have the full permission to use the data, it is very recent (we get updates
> monthly) and quite precise: all locations are manually tweaked after
> geocoding. It has fuel stations of major brands (Shell, BP, Total) and
> smaller, non-chain owners (ProxiFuel, Lotherol, Edeka).
>
> You can see the numbers and check a few stations yourself at the
> validation website (it will be very slow, please bear with it):
>
> http://audit.osmz.ru/project/navads_fuel
>
> I want to import this data to OpenStreetMap. Which means, to update tags
> on 38k fuel stations and add 21k stations not present on OSM. For that, I
> need you to have a look at the data. Of course it is impossible to manually
> check every object, and to do that afterwards each month up to eternity.
> But you can help check that tags are correct, that there are no duplicates
> being created, and that postcodes on objects are good to import.
>
> The source dataset has much more information than I upload. For example,
> it has flags for grocery stores, car washes, restaurants in fuel station
> buildings, and I don't know tags for these. It has street addresses, but
> they need to be parsed, and that isn't easy ("62 Rue Du General Leclerc",
> "Pritzwalker Straße 20", "Allée de Provence Bld Des Alpes De Haute
> Provence"). It has town and state and country for each fuel station. You
> can see a sample of the source json here, and maybe suggest on using some
> of the extra information:
>
> https://pastebin.com/SLXq8Eb8
>
> Please be gentle. I know some of you despise large-scale imports, but this
> one to me is as good as imports get. I'd like to get it to completion, even
> if all that remains from the source data are the type and the brand name.
>
> Thanks,
> Ilya
> ___
> Imports mailing list
> Imports@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>



-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports

[Imports] Worldwide fuel stations import, 59k objects

[Ilya Zverev]

Hi everyone,

Following the recent UK Shell stations import, I've got ahold of the entire 
NavAds dataset. A major part of it are fuel stations all across the world: UK, 
US, France, Germany, Australia, and many other countries. We have the full 
permission to use the data, it is very recent (we get updates monthly) and 
quite precise: all locations are manually tweaked after geocoding. It has fuel 
stations of major brands (Shell, BP, Total) and smaller, non-chain owners 
(ProxiFuel, Lotherol, Edeka).

You can see the numbers and check a few stations yourself at the validation 
website (it will be very slow, please bear with it):

http://audit.osmz.ru/project/navads_fuel

I want to import this data to OpenStreetMap. Which means, to update tags on 38k 
fuel stations and add 21k stations not present on OSM. For that, I need you to 
have a look at the data. Of course it is impossible to manually check every 
object, and to do that afterwards each month up to eternity. But you can help 
check that tags are correct, that there are no duplicates being created, and 
that postcodes on objects are good to import.

The source dataset has much more information than I upload. For example, it has 
flags for grocery stores, car washes, restaurants in fuel station buildings, 
and I don't know tags for these. It has street addresses, but they need to be 
parsed, and that isn't easy ("62 Rue Du General Leclerc", "Pritzwalker Straße 
20", "Allée de Provence Bld Des Alpes De Haute Provence"). It has town and 
state and country for each fuel station. You can see a sample of the source 
json here, and maybe suggest on using some of the extra information:

https://pastebin.com/SLXq8Eb8

Please be gentle. I know some of you despise large-scale imports, but this one 
to me is as good as imports get. I'd like to get it to completion, even if all 
that remains from the source data are the type and the brand name.

Thanks,
Ilya
___
Imports mailing list
Imports@openstreetmap.org
https://lists.openstreetmap.org/listinfo/imports