Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Kevin Kenny]

On Thu, Nov 30, 2017 at 2:30 PM, Yuri Astrakhan  wrote:
> Erkin, the whole idea of the permanent ID is for it to always point to the
> same "conceptual" object. If I create a road, and use an ID for that road
> somewhere, I would like that ID to continue working even if the road gets
> broken up into multiple segments.  I am not exactly sure how your approach
> would solve that.

I really favour a "good enough" solution over a "perfect" one.

The existing wikidata=* scheme is 'pretty good' in that while it won't do things
like "search for all public art works by a given artist", it will
allow backlinking
to specific objects in OSM by means of an Overpass query.

Few mappers, when casually splitting a way, simply delete tags willy-nilly.
Moreover, most way splitting happens in order to do things like add part of
the way to a route relation, put a traffic restriction on part of the
way, or other
things that don't change the 'identity' of the given way, so a 'wikidata=*'
tag is highly likely to be conserved. (If mappers simply replace one way
with another, obviously all bets are off, but that's already regarded as
impolite, particularly if they destroy existing tagging while doing so.)

If we abandon the idea that the back link must be to a single OSM object
and instead think of it as a link to a set of objects (ideally singleton, but
possibly zero, or more than one), to what extent does the problem go
away? The data link in both directions, and the tag is likely to be conserved
even when the underlying objects are restructured.

Is this at least the 90% solution? Will it buy us time and the opportunity
to grow into a better answer? What am I missing?

For what it's worth, I already maintain a few sets of [mostly imported]
objects in the database that bear tags whose values are
ID's in several foreign databases. "Any tag you like."
They are there precisely so that I can use them as forward links
and use Overpass queries as back links. I don't demand that they
be 100% stable, just that I have some likelihood of finding again an
object that I labeled once before. If mappers mess up a few of them,
it's no big deal, I don't expect crowdsourced data to follow any
external schema rigorously. Just as with anything else on the Web,
broken links happen sometimes. If the tags were deleted wholesale,
that would make me quite cross, particularly since we have no better
theory about how to deal (even approximately) with references between
our data and data housed in other repositories.

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Erkin Alp Güney]

30-11-2017 22:56 tarihinde Yuri Astrakhan yazdı:
>
> If you edit a road, a new one would be created and would point to its
> invalidated ancestor. Recursively chasing previous ID pointers, you
> would eventually have an object without an ancestor. ID of that object
> would also be permanent ID of the successor objects. This will also
> solve road split problems as multiple objects can point to the same
> ancestor.
>
>
> Erikn, if I understood you correctly, you are proposing the data
> structure to generate and track IDs.  I think there are several issues:
> * determining feature ID requires O(N) - linear search through history.
If you just want the latest version, one lookup. If you want ancestors
and descendants, linear pointer chase.
> * given an ID, if you want to examine what it points to, you have to
> assemble the whole tree of descendants to determine all of the leafs,
> and test if they still exist or were deleted. Which also means that
> instead of a human-currated single relation for the multi-segment
> road, and a single list of tags, you are now looking at a set of
> objects, each with its own list of tags.
ID-only lean tree may be stored at the server side. You will then only
download the objects you actually want.
> * Download must contain an entire OSM change history to work with the
> IDs offline
>
See git's shallow cloning. As OSM IDs are linear counter rather than
hashes, it is even easier to work: You can prune objects by ID thresholding.

Yours, faithfully
Erkin Alp

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Yuri Astrakhan]

> If you edit a road, a new one would be created and would point to its
> invalidated ancestor. Recursively chasing previous ID pointers, you
> would eventually have an object without an ancestor. ID of that object
> would also be permanent ID of the successor objects. This will also
> solve road split problems as multiple objects can point to the same
> ancestor.
>

Erikn, if I understood you correctly, you are proposing the data structure
to generate and track IDs.  I think there are several issues:
* determining feature ID requires O(N) - linear search through history.
* given an ID, if you want to examine what it points to, you have to
assemble the whole tree of descendants to determine all of the leafs, and
test if they still exist or were deleted. Which also means that instead of
a human-currated single relation for the multi-segment road, and a single
list of tags, you are now looking at a set of objects, each with its own
list of tags.
* Download must contain an entire OSM change history to work with the IDs
offline
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[marc marc]

Le 30. 11. 17 à 20:30, Yuri Astrakhan a écrit :
> If I create a road, and use an ID for that road somewhere, I would like 
> that ID to continue working even if the road gets broken up into 
> multiple segments.

why not use the existing overpass features ?
you can define which criteria determine whether or not an object is 
always the same in the event of a change.
For example :
- a road in a city with such a name. If the way is divided, merged, 
delete+create, it will always have the same query id.
if the way is cut in 2 with a part that changes its name, this part will 
no longer be linked to the id of the previous request.
- In case of poi in a building, you can easily create an id for the 
building and another id for the poi.
- a poi that can close or move : you can make a request id related to 
the poi at this specific location (and which will correspond to the new 
business if the previous one closes) or you can choose if the id 
corresponds to the poi even if it moves to the next street.

Or course mabe we can create a OsmData that host and/or redirect to 
overpass query. but it'll avoid the need to make software change at a 
large-scale to work.

Regards,
Marc
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Erkin Alp Güney]

If you edit a road, a new one would be created and would point to its
invalidated ancestor. Recursively chasing previous ID pointers, you
would eventually have an object without an ancestor. ID of that object
would also be permanent ID of the successor objects. This will also
solve road split problems as multiple objects can point to the same
ancestor.


30-11-2017 22:30 tarihinde Yuri Astrakhan yazdı:
>
> Immutable objects with a previous ID field would solve that. Every
> edit
> will create or delete, no modify. First version's ID will be your
> persistent ID.
>
>
> Erkin, the whole idea of the permanent ID is for it to always point to
> the same "conceptual" object. If I create a road, and use an ID for
> that road somewhere, I would like that ID to continue working even if
> the road gets broken up into multiple segments.  I am not exactly sure
> how your approach would solve that.
Yours, faithfully
Erkin Alp


___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Yuri Astrakhan]

>
> Immutable objects with a previous ID field would solve that. Every edit
> will create or delete, no modify. First version's ID will be your
> persistent ID.
>

Erkin, the whole idea of the permanent ID is for it to always point to the
same "conceptual" object. If I create a road, and use an ID for that road
somewhere, I would like that ID to continue working even if the road gets
broken up into multiple segments.  I am not exactly sure how your approach
would solve that.
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Erkin Alp Güney]

Yuri Astrakhan wrote:

> Implementing the permanent ID is conceptually (relatively) simple, but
> would require a lot of work.  "When to break continuity" seems to be
> the philosophical sticking point, as pointed out by others in this
> discussion.  We could agree on some general continuity guidelines and
> let it be solved by each editor on a case-by-case basis.  It will
> never be perfect, but neither is not doing anything.
>
Immutable objects with ancestor pointer fields are far easier to
implement than implementing a Wikidata analog for OSM.

Yours, faithfully
Erkin Alp

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Yuri Astrakhan]

>
> I think permanent IDs should be done with our own Wikidata. Wikibase is a
> Wikimedia extension, that means our own Wiki can install OSMData right now.
> Then if you want to open a permanent ID for a shop, create a new OSMData
> item, and tag the shop with OSMData=M38267 (M instead of Q, for Map).
>

Janko, I'm all for using a better storage tech if warranted, but I don't
think introducing Wikibase will solve the problem mentioned by others --
**what is an entity?**. Wikidata community is all about defining those
conceptual entities, and OSM community is mostly concerned with reflecting
the current state of the world.  In other words, OSM community does not
concentrate with the historical continuity -- preserving the history of a
"concept", or even specifying the definition of the "concept".

The two basic continuity questions need to be solved for permanent IDs to
become a reality:
* keeping an ID when feature's geometry change - e.g. a single road way
becomes multiple segments, merging duplicates with redirects, etc.
* deciding when to break the continuity when the feature is "no longer the
same concept" - e.g. a restaurant closed, and a new one opened in its place
needs a new ID, instead of reusing the same node.

Implementing the permanent ID is conceptually (relatively) simple, but
would require a lot of work.  "When to break continuity" seems to be the
philosophical sticking point, as pointed out by others in this discussion.
We could agree on some general continuity guidelines and let it be solved
by each editor on a case-by-case basis.  It will never be perfect, but
neither is not doing anything.
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Janko Mihelić]

I think permanent IDs should be done with our own Wikidata. Wikibase is a
Wikimedia extension, that means our own Wiki can install OSMData right now.
Then if you want to open a permanent ID for a shop, create a new OSMData
item, and tag the shop with OSMData=M38267 (M instead of Q, for Map).

This permanent ID won't just be an empty number, but it can have all sorts
of structured data attached to it, for example Name, so mappers can see
what that ID is about.

Of course, duplicating data would be strongly discouraged, and that can be
easily done by restricting properties that can be added to items. So
property "religion" won't be available for church items, because that
property is already expected in the OSM database.

Janko
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Christoph Hormann]

On Thursday 30 November 2017, Frederik Ramm wrote:
>
> Biggest issue IMHO that came up in the UUID discussion is that it is
> unclear what the ID relates to, and mappers cannot be expected to
> handle that topic in all its complexity properly.

Note essentially the same applies to wikidata tags - just that the 
problems are seemingly outsourced to an external project and the mapper 
does not have to deal with them - which is wrong of course.

-- 
Christoph Hormann
http://www.imagico.de/

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Martin Koppenhoefer]

2017-11-30 7:42 GMT+01:00, Yuri Astrakhan :
>> * What if the shop moves to a new address ? What if someone already
>> recorded the shop at the new location, how do we merge the IDs?
> If we decide that a moved shop should have the same id, than this is a case
> for merge+making one redirect to the other.


what if the shop moves to 2 different addresses? In this case the ID
wouldn't be unique any more (or we'd have to create a relation to hold
the ID, i.e. change our mapping technique due to the ID concept, i.e.
more complexity).


>> * A restaurant POI and the building it resides in are 2 different
>> concepts, they both need their own permanent ID. How can we
>> distinguish them ?
>
> If the entire building is a restaurant (e.g. a standalone McDonald's), than
> it should probably be the same id. But if restaurant is occupying the first
> floor of a residential bldg, probably two.


No, I don't think it is a good idea to have the same ID, because many
properties will still be different (e.g. start date, architect of shop
will often be different to that of the building, operator might be
different, etc.) and many properties will only apply to one (e.g. a
building doesn't have a "cuisine" in the OSM tag sense).


> How does the software knows this when someone "joins" the POI and the
>> building ?
> Software wouldn't know, but an editor should always keep this in mind, and
> there tools should offer some "join" mechanism.


IMHO buildings and users of the building should in most cases be distinguished.



>> * What do you do when a modern part is attached to a listed building ?
>> Will the building parts have permanent IDs ?
> If the building or a road is extended, it should keep the old id. If the
> new part is notable on its own, the part may get a new id as well, while
> also being part of the old id.


"notable"? That wasn't an OSM concept until now. IMHO we would have to
have a mechanism that keeps trace of what happens to IDs (e.g. this ID
is now part of this ID, or has become this ID, or was deleted because
of a / b / c / d etc.)

Ultimately, we'll need an ID for every tag, and for every combination
of tags, of an object (i.e. a lot of IDs for every object). So every
need can be suited ;-)

I still believe OSM IDs together with versions / timestamps, are
already suitable for referring to a certain state at a given point in
time. Additional IDs introduce more problems than they promise to
solve (because mappers will have to decide what to do with them when
modifying the map, and therefore the IDs would either have to be very
finegrained (as stated above, one for every tag, and for every
combination of tags), or would sooner or later brake for some of the
usecases for which people rely on them (e.g. one mapper seeing the ID
referring to the restaurant, the other to the building).


Cheers,
Martin

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Erkin Alp Güney]

Immutable objects with a previous ID field would solve that. Every edit
will create or delete, no modify. First version's ID will be your
persistent ID.


30-11-2017 10:28 tarihinde Frederik Ramm yazdı:
> Ah, I forgot to say: The result of many, lengthy, previous discussions
> about permanent IDs was generally: We don't want to shoulder the
> responsibility of maintaining a permanent ID in OSM that others can use
> for easy linking; instead, those others should be doing that work for
> themselves. Hence the idea of "fuzzy" permanent links was born, where
> your link is not really a link but a search query:


___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Frederik Ramm]

Ah, I forgot to say: The result of many, lengthy, previous discussions
about permanent IDs was generally: We don't want to shoulder the
responsibility of maintaining a permanent ID in OSM that others can use
for easy linking; instead, those others should be doing that work for
themselves. Hence the idea of "fuzzy" permanent links was born, where
your link is not really a link but a search query:

"A restaurant named Bella Italia in the general vicinity of X", or "The
restaurant that is in this building, if any", and so on. That way, the
person placing the link has to decide which property, for *them*, is the
"essence" of what they're pointing to, instead of burdening us in OSM to
add a ton of IDs to a restaurant object. Some work has been done by
Roland Olbricht (of Overpass API) to make the concept workable, see:

http://wiki.openstreetmap.org/wiki/Overpass_API/Permanent_ID

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Frederik Ramm]

Hi,

On 30.11.2017 03:13, Yuri Astrakhan wrote:
> Permanent IDs has been brought up several times, especially as part of
> the Wikidata ID discussion. I started a wiki page to outline the
> requirements and goals, but it might be incomplete, feel free to add /
> correct / comment.

See https://wiki.openstreetmap.org/wiki/Proposed_Features/UUID for an
earlier and somewhat failed idea.

Biggest issue IMHO that came up in the UUID discussion is that it is
unclear what the ID relates to, and mappers cannot be expected to handle
that topic in all its complexity properly.

Example:

Restaurant "Bella Italia" with well-known Michelin 4-star chef changes
its name to "Apulia" but otherwise stays the same. What happens with the
ID? What if someone has, somewhere, compiled a list: "All restaurants
named something with Italy" and linked to them by ID?

Example:

Same case as above, but restaurant (including chef and name) moves to a
different part of the city. Should the ID move? What if someone has
compiled a list: "Restaurants in 19th century buildings", of which the
old "Bella Italia" was a part, but the new one isn't?

To do this right, you would have to have different IDs for the same OSM
object - one for the "instance of restaurant" that happens to be here,
one for the building, one for the chef, one for the name - there's no
end. Somehow with a permanent ID you want to capture the "essence" of
something and stick the ID to that, but the "essence" of something is
often a combination of factors (name, location, cuisine...).

There are many other examples that illustrate that the very concept of
permanent IDs is hard to make work in OSM.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Yuri Astrakhan]

Marc, good points, thanks. Naturally it will never be decided by software.
The tools should aid in continuity preservation, not dictate it. We as a
community should definitely how each one of these and other cases should be
handled -- to define what "permanence" means.

>

> * What if the shop moves to a new address ? What if someone already
> recorded the shop at the new location, how do we merge the IDs?
>
If we decide that a moved shop should have the same id, than this is a case
for merge+making one redirect to the other.

>
> * A restaurant POI and the building it resides in are 2 different
> concepts, they both need their own permanent ID. How can we
> distinguish them ?

If the entire building is a restaurant (e.g. a standalone McDonald's), than
it should probably be the same id. But if restaurant is occupying the first
floor of a residential bldg, probably two.

How does the software knows this when someone "joins" the POI and the
> building ?
>
Software wouldn't know, but an editor should always keep this in mind, and
there tools should offer some "join" mechanism.

>

> * What do you do when a modern part is attached to a listed building ?
> Will the building parts have permanent IDs ?
>
If the building or a road is extended, it should keep the old id. If the
new part is notable on its own, the part may get a new id as well, while
also being part of the old id.

>
> This is something you cannot solve with software alone. A human has to
> try to decide what you have to do with the permanent IDs in some
> (many?) cases.
>
I would say the human has to decide it in every case :). The software
should help with the process. That's why I listed community education as a
big part of introducing this.

Btw, permanent IDs would help with historical maps too.
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

Re: [Tagging] Permanent IDs RFC (was part_of:wikidata)

[Marc Gemis]

I also see several problems to define the "same" concept:

* a shop that closes (bankrupt) and a new that opens afterwards. Is
this the same shop ? Is there a difference when the shop owner remains
the same?  When the products they sell are the same (I have a case of
bakery that closed and opened a few months later with new owners and a
different name in my town) ?

* What if the shop moves to a new address ? What if someone already
recorded the shop at the new location, how do we merge the IDs?

* A restaurant POI and the building it resides in are 2 different
concepts, they both need their own permanent ID. How can we
distinguish them ? How does the software knows this when someone
"joins" the POI and the building ?

* What do you do when a modern part is attached to a listed building ?
Will the building parts have permanent IDs ?

This is something you cannot solve with software alone. A human has to
try to decide what you have to do with the permanent IDs in some
(many?) cases.

m.

On Thu, Nov 30, 2017 at 3:13 AM, Yuri Astrakhan  wrote:
> Permanent IDs has been brought up several times, especially as part of the
> Wikidata ID discussion. I started a wiki page to outline the requirements
> and goals, but it might be incomplete, feel free to add / correct / comment.
>
> https://wiki.openstreetmap.org/wiki/Permanent_ID
>
> Once we reach the agreement on the goals, we can figure out the
> implementation strategy.
>
>
> On Tue, Nov 28, 2017 at 2:47 PM, Andy Mabbett 
> wrote:
>>
>> On 28 November 2017 at 16:40, Christoph Hormann  wrote:
>>
>> > The problem is OSM is a map of the physical world, not a map of the
>> > world's databases.  If Wikidata wants to create links between OSM and
>> > other databases that is great but so far i think no one has made a good
>> > case why this linking information should be stored in OSM rather than
>> > Wikidata.
>>
>> Then you are not paying attention. OSM IDs are volatile - far more
>> volatile than Wikipedia IDs, let alone Wikidata IDs.
>>
>> > Again my suggestion: Working on better ways to address features in OSM
>> > in a stable way from the outside would be much more productive
>>
>> Great! Let us know when you have a working solution, consensus to
>> implement it, and tools that work with it.
>>
>> --
>> Andy Mabbett
>> @pigsonthewing
>> http://pigsonthewing.org.uk
>>
>> ___
>> Tagging mailing list
>> Tagging@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/tagging
>
>
>
> ___
> Tagging mailing list
> Tagging@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/tagging
>

___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging

[Tagging] Permanent IDs RFC (was part_of:wikidata)

[Yuri Astrakhan]

Permanent IDs has been brought up several times, especially as part of the
Wikidata ID discussion. I started a wiki page to outline the requirements
and goals, but it might be incomplete, feel free to add / correct / comment.

https://wiki.openstreetmap.org/wiki/Permanent_ID

Once we reach the agreement on the goals, we can figure out the
implementation strategy.


On Tue, Nov 28, 2017 at 2:47 PM, Andy Mabbett 
wrote:

> On 28 November 2017 at 16:40, Christoph Hormann  wrote:
>
> > The problem is OSM is a map of the physical world, not a map of the
> > world's databases.  If Wikidata wants to create links between OSM and
> > other databases that is great but so far i think no one has made a good
> > case why this linking information should be stored in OSM rather than
> > Wikidata.
>
> Then you are not paying attention. OSM IDs are volatile - far more
> volatile than Wikipedia IDs, let alone Wikidata IDs.
>
> > Again my suggestion: Working on better ways to address features in OSM
> > in a stable way from the outside would be much more productive
>
> Great! Let us know when you have a working solution, consensus to
> implement it, and tools that work with it.
>
> --
> Andy Mabbett
> @pigsonthewing
> http://pigsonthewing.org.uk
>
> ___
> Tagging mailing list
> Tagging@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/tagging
>
___
Tagging mailing list
Tagging@openstreetmap.org
https://lists.openstreetmap.org/listinfo/tagging