Re: [Talk-transit] Uploading public transport data on OSM

[john whelan]

Within a geographic area could we accept that all bus stops with a specific
tag were GTFS tags for a particular transit company?

Thanks John

On 16 Jan 2018 7:42 pm, "Johnparis"  wrote:

> I believe OSM-Sync creates nodes with "gtfs_id" as the tag key. The value
> is typically something like "StopPoint:48:5001".
>
> In response to Jo/Polyglot's concern, the gtfs_id is not unique globally;
> it is unique within a GTFS feed. So, for instance, the Paris area's
> transport agency, STIF (now IDFM), has unique IDs within the Paris region.
> It is theoretically possible that these could be duplicated somewhere else
> in the world (though that likelihood is extremely small, given the
> numbering scheme they use). The STIF scheme is StopPoint:x:y, where x is
> the (internal STIF) code of the agency providing the stop times for each
> trip, and y is an arbitrary number guaranteed to be unique within the
> agency. (The agency codes are in the agency.txt file in the GTFS feed. It
> gets quite arbitrary; for instance the RATP, which runs the Paris transport
> system, has an agency code of 59 for purposes of bus StopPoints but a code
> of 100 for some other purposes. Weird.)
>
> I agree with Stephen Sprunk about the need to sync with a unique ID. I
> have begun using gtfs_id (easily changed to, for example,
> ref:FR:STIF:gtfs_id), with the idea that changes would first be synched
> against the existing ID. STIF does not guarantee that the gtfs_id for a
> stop is permanent; however, it seems to be the case. (STIF did guarantee
> that a different code would be permanent, but it seems to have abandoned
> that recently, and GTFS is becoming the de facto standard.) It makes my
> life a little simpler to use gtfs_id instead of ref:FR:STIF:gtfs_id because
> the matching logic is simpler. (I started with ref:FR:STIF:gtfs_id but then
> in JOSM for instance I could not do a search for "gtfs_id -ref" because the
> "-ref" includes any ref, even ref:FR:STIF:gtfs_id. Using just gtfs_id
> avoids that problem.)
>
> The problem I run into most frequently is when someone "on the ground"
> (that is, a mapper) has created a bus stop (usually with a position much
> more accurate than the agency's data) but the stop doesn't have useful
> information to derive the gtfs_id. (For instance, if the node had an
> accurate stop name and the number of a bus line that serves it, I could
> derive the gtfs_id, but often there isn't anything more informative than
> "highway=bus_stop".) As a result, I have been importing nodes for each bus
> line from the STIF data (with permission), then manually merging the nodes
> with those already in place on the ground. I save myself lots of time when
> I encounter a new line that uses existing gtfs_ids.
>
> I have a couple of thoughts about GSoC projects for JOSM plugins that
> might be useful.
>
> -- One would deal with the aforementioned node-merging problem (an
> interesting theoretical problem; mostly you want to match the closest stop
> on the same side of the road, assuming it's within a reasonable distance).
>
> -- Another would be a route-builder. My idea would be (a) I provide a
> starting way and direction, (b) at the end of each way, I indicate left,
> right, straight or other to continue. I think this would greatly speed the
> creation of routes.
>
> Perhaps these tools already exist. I'd love to see them.
>
> John
>
>
>
>>
>>
>> --
>>
>> Message: 5
>> Date: Tue, 16 Jan 2018 21:35:54 +0100
>> From: Jo 
>> To: "Public transport/transit/shared taxi related topics"
>> 
>> Subject: Re: [Talk-transit] Uploading public transport data on OSM
>> Message-ID:
>> > ail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Is there a common pattern to these GTFS IDs? Are they guaranteed to be
>> unique across operators?
>>
>> Or is that not important and is it only important that they are stable
>> between versions of GTFS files for a region?
>>
>> Adding a ref:gtfs tag would not be very hard to do, but I would prefer a
>> scheme with ref:OPERATOR, because some stops may be served by multiple
>> operators and thus be present in multiple GTFS feeds.
>>
>> Polyglot
>>
>> 2018-01-16 21:07 GMT+01:00 Stephen Sprunk :
>>
>> >
>> >
>> > What I'd like to see is some sort of tag on OSM objects (stops, routes,
>> > etc.) listing the GTFS ID numbers so that tools can more easily connect
>> the
>> > two; that should be easy enough if someone defines a scheme and gets the
>> > few relevant tools to use it.
>> >
>> > The lat/long for stops in GTFS data is often questionable, so it would
>> be
>> > good to have some way for folks to be able to fix the stop locations in
>> OSM
>> > and not get overwritten by another import later.  In many places
>> > (especially in the US), GTFS data will change every few months, so if we
>> > 

Re: [Talk-transit] Uploading public transport data on OSM

[Johnparis]

I believe OSM-Sync creates nodes with "gtfs_id" as the tag key. The value
is typically something like "StopPoint:48:5001".

In response to Jo/Polyglot's concern, the gtfs_id is not unique globally;
it is unique within a GTFS feed. So, for instance, the Paris area's
transport agency, STIF (now IDFM), has unique IDs within the Paris region.
It is theoretically possible that these could be duplicated somewhere else
in the world (though that likelihood is extremely small, given the
numbering scheme they use). The STIF scheme is StopPoint:x:y, where x is
the (internal STIF) code of the agency providing the stop times for each
trip, and y is an arbitrary number guaranteed to be unique within the
agency. (The agency codes are in the agency.txt file in the GTFS feed. It
gets quite arbitrary; for instance the RATP, which runs the Paris transport
system, has an agency code of 59 for purposes of bus StopPoints but a code
of 100 for some other purposes. Weird.)

I agree with Stephen Sprunk about the need to sync with a unique ID. I have
begun using gtfs_id (easily changed to, for example, ref:FR:STIF:gtfs_id),
with the idea that changes would first be synched against the existing ID.
STIF does not guarantee that the gtfs_id for a stop is permanent; however,
it seems to be the case. (STIF did guarantee that a different code would be
permanent, but it seems to have abandoned that recently, and GTFS is
becoming the de facto standard.) It makes my life a little simpler to use
gtfs_id instead of ref:FR:STIF:gtfs_id because the matching logic is
simpler. (I started with ref:FR:STIF:gtfs_id but then in JOSM for instance
I could not do a search for "gtfs_id -ref" because the "-ref" includes any
ref, even ref:FR:STIF:gtfs_id. Using just gtfs_id avoids that problem.)

The problem I run into most frequently is when someone "on the ground"
(that is, a mapper) has created a bus stop (usually with a position much
more accurate than the agency's data) but the stop doesn't have useful
information to derive the gtfs_id. (For instance, if the node had an
accurate stop name and the number of a bus line that serves it, I could
derive the gtfs_id, but often there isn't anything more informative than
"highway=bus_stop".) As a result, I have been importing nodes for each bus
line from the STIF data (with permission), then manually merging the nodes
with those already in place on the ground. I save myself lots of time when
I encounter a new line that uses existing gtfs_ids.

I have a couple of thoughts about GSoC projects for JOSM plugins that might
be useful.

-- One would deal with the aforementioned node-merging problem (an
interesting theoretical problem; mostly you want to match the closest stop
on the same side of the road, assuming it's within a reasonable distance).

-- Another would be a route-builder. My idea would be (a) I provide a
starting way and direction, (b) at the end of each way, I indicate left,
right, straight or other to continue. I think this would greatly speed the
creation of routes.

Perhaps these tools already exist. I'd love to see them.

John



>
>
> --
>
> Message: 5
> Date: Tue, 16 Jan 2018 21:35:54 +0100
> From: Jo 
> To: "Public transport/transit/shared taxi related topics"
> 
> Subject: Re: [Talk-transit] Uploading public transport data on OSM
> Message-ID:
>  gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Is there a common pattern to these GTFS IDs? Are they guaranteed to be
> unique across operators?
>
> Or is that not important and is it only important that they are stable
> between versions of GTFS files for a region?
>
> Adding a ref:gtfs tag would not be very hard to do, but I would prefer a
> scheme with ref:OPERATOR, because some stops may be served by multiple
> operators and thus be present in multiple GTFS feeds.
>
> Polyglot
>
> 2018-01-16 21:07 GMT+01:00 Stephen Sprunk :
>
> >
> >
> > What I'd like to see is some sort of tag on OSM objects (stops, routes,
> > etc.) listing the GTFS ID numbers so that tools can more easily connect
> the
> > two; that should be easy enough if someone defines a scheme and gets the
> > few relevant tools to use it.
> >
> > The lat/long for stops in GTFS data is often questionable, so it would be
> > good to have some way for folks to be able to fix the stop locations in
> OSM
> > and not get overwritten by another import later.  In many places
> > (especially in the US), GTFS data will change every few months, so if we
> > want it in OSM at all, we need a scheme that can deal with regular bulk
> > imports without losing quality where local OSM editors have improved
> things
> > by hand.
> >
> > S
> >
> > --
> > Stephen Sprunk  "Those people who think they know everything
> > CCIE #3723 are a great annoyance to those of us who do."
> > K5SSS 

Re: [Talk-transit] Uploading public transport data on OSM

[Jo]

That is why I think it's important not to go with ref:gtfs, but use
ref:operator1, ref:operator2. Where operator1 and operator2 are the names
or short names for the different operators. In many cases you'll find that
these refs are also what the public sees on the stops (if the operators
decide to put it there) or in internet urls when using the operator's
website.

Here in Belgium, we have 3 operators extending into the other operator's
mostly served area and some lines extend into neighbouring countries, so we
had to namespace the ref tag a few years ago, already.

Polyglot

2018-01-16 22:07 GMT+01:00 Stephen Sprunk :

> AFAIK, the operator ID is only guaranteed to be unique within a
> single GTFS feed, but it's reasonably safe to assume they'll also be unique
> between feeds with overlapping geographical areas.  It's probably not safe
> to assume that's transitive.
>
> Where operators don't cooperate and produce a single joint feed, you'll
> inevitably face the "same" stops appearing in multiple feeds.  Imagine a
> single roadside bus stop pole with signs from two or three different bus
> operators on it, and that pole would have a different ID and probably even
> different lat/long in each feed.  Ideally, a local OSM editor would be able
> to merge them into a single stop object--and that merger would survive
> later bulk imports from all those GTFS feeds.  This would cut down on map
> clutter and allow routing apps to more easily recognize transfers, but I'm
> not sure how to design such a schema.
>
> It's been a while since I read the spec, but I think GTFS also has a
> similar concept to stop_areas, and that will probably have similar problems
> to stops.
>
> Routes appear simpler to deal with since they're unique to each operator,
> but they're inherently built on top of stops (or stop_areas), so they may
> present new problems when we get there.
>
> S
>
>
> On 2018-01-16 14:35, Jo wrote:
>
> Is there a common pattern to these GTFS IDs? Are they guaranteed to be
> unique across operators?
>
> Or is that not important and is it only important that they are stable
> between versions of GTFS files for a region?
>
> Adding a ref:gtfs tag would not be very hard to do, but I would prefer a
> scheme with ref:OPERATOR, because some stops may be served by multiple
> operators and thus be present in multiple GTFS feeds.
>
> Polyglot
>
> 2018-01-16 21:07 GMT+01:00 Stephen Sprunk :
>
>> On 2018-01-16 08:30, Mike N wrote:
>>
>>> On 1/16/2018 7:37 AM, Yash Ganthe wrote:
>>>
 What caught my attention is a project called Go Sync
 https://wiki.openstreetmap.org/wiki/GO-Sync
 The description indicates that it is for syncing GTFS with OSM. But I
 am not sure what type of account is needed for that.
>>>
>>>
>>>
>>>  From what I recall, Go-Sync only synchronizes stops but not the GTFS
>>> route paths with OSM route relations.
>>>
>>>  The routing samples on the OpenStreetMap web site will not
>>> automatically give public transport trip routing because the full GTFS
>>> schedule information is not in OSM.   It would require a separate
>>> local site such as OpenTripPlanner which automatically combines full
>>> GTFS with OSM data.   There are some companies providing large scale
>>> instances of OpenTripPlanner which may cover your area.
>>
>> What I'd like to see is some sort of tag on OSM objects (stops, routes,
>> etc.) listing the GTFS ID numbers so that tools can more easily connect the
>> two; that should be easy enough if someone defines a scheme and gets the
>> few relevant tools to use it.
>>
>> The lat/long for stops in GTFS data is often questionable, so it would be
>> good to have some way for folks to be able to fix the stop locations in OSM
>> and not get overwritten by another import later.  In many places
>> (especially in the US), GTFS data will change every few months, so if we
>> want it in OSM at all, we need a scheme that can deal with regular bulk
>> imports without losing quality where local OSM editors have improved things
>> by hand.
>>
>> S
>>
>> --
>> Stephen Sprunk  "Those people who think they know everything
>> CCIE #3723 are a great annoyance to those of us who do."
>> K5SSS --Isaac Asimov
>>
>>
>> ___
>> Talk-transit mailing list
>> Talk-transit@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-transit
>>
>
> ___
> Talk-transit mailing list
> Talk-transit@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-transit
>
>
> --
> Stephen Sprunk  "Those people who think they know everything
> CCIE #3723 are a great annoyance to those of us who do."
> K5SSS --Isaac Asimov
>
___
Talk-transit mailing list
Talk-transit@openstreetmap.org

Re: [Talk-transit] Uploading public transport data on OSM

[Stephen Sprunk]

AFAIK, the operator ID is only guaranteed to be unique within a single
GTFS feed, but it's reasonably safe to assume they'll also be unique
between feeds with overlapping geographical areas.  It's probably not
safe to assume that's transitive. 

Where operators don't cooperate and produce a single joint feed, you'll
inevitably face the "same" stops appearing in multiple feeds.  Imagine a
single roadside bus stop pole with signs from two or three different bus
operators on it, and that pole would have a different ID and probably
even different lat/long in each feed.  Ideally, a local OSM editor would
be able to merge them into a single stop object--and that merger would
survive later bulk imports from all those GTFS feeds.  This would cut
down on map clutter and allow routing apps to more easily recognize
transfers, but I'm not sure how to design such a schema. 

It's been a while since I read the spec, but I think GTFS also has a
similar concept to stop_areas, and that will probably have similar
problems to stops. 

Routes appear simpler to deal with since they're unique to each
operator, but they're inherently built on top of stops (or stop_areas),
so they may present new problems when we get there. 

S 

On 2018-01-16 14:35, Jo wrote:

> Is there a common pattern to these GTFS IDs? Are they guaranteed to be unique 
> across operators?
> 
> Or is that not important and is it only important that they are stable 
> between versions of GTFS files for a region?
> 
> Adding a ref:gtfs tag would not be very hard to do, but I would prefer a 
> scheme with ref:OPERATOR, because some stops may be served by multiple 
> operators and thus be present in multiple GTFS feeds.
> 
> Polyglot 
> 
> 2018-01-16 21:07 GMT+01:00 Stephen Sprunk :
> On 2018-01-16 08:30, Mike N wrote:
> On 1/16/2018 7:37 AM, Yash Ganthe wrote:
> What caught my attention is a project called Go Sync 
> https://wiki.openstreetmap.org/wiki/GO-Sync [1]
> The description indicates that it is for syncing GTFS with OSM. But I am not 
> sure what type of account is needed for that. 
> 
> From what I recall, Go-Sync only synchronizes stops but not the GTFS
> route paths with OSM route relations.
> 
> The routing samples on the OpenStreetMap web site will not
> automatically give public transport trip routing because the full GTFS
> schedule information is not in OSM.   It would require a separate
> local site such as OpenTripPlanner which automatically combines full
> GTFS with OSM data.   There are some companies providing large scale
> instances of OpenTripPlanner which may cover your area.
 What I'd like to see is some sort of tag on OSM objects (stops, routes,
etc.) listing the GTFS ID numbers so that tools can more easily connect
the two; that should be easy enough if someone defines a scheme and gets
the few relevant tools to use it.

The lat/long for stops in GTFS data is often questionable, so it would
be good to have some way for folks to be able to fix the stop locations
in OSM and not get overwritten by another import later.  In many places
(especially in the US), GTFS data will change every few months, so if we
want it in OSM at all, we need a scheme that can deal with regular bulk
imports without losing quality where local OSM editors have improved
things by hand.

S

-- 
Stephen Sprunk  "Those people who think they know everything
CCIE #3723 are a great annoyance to those of us who do."
K5SSS --Isaac Asimov 

___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit [2] 
___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit 

-- 
Stephen Sprunk  "Those people who think they know everything
CCIE #3723 are a great annoyance to those of us who do."
K5SSS --Isaac Asimov 

Links:
--
[1] https://wiki.openstreetmap.org/wiki/GO-Sync
[2] https://lists.openstreetmap.org/listinfo/talk-transit___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit

Re: [Talk-transit] Uploading public transport data on OSM

[Jo]

Is there a common pattern to these GTFS IDs? Are they guaranteed to be
unique across operators?

Or is that not important and is it only important that they are stable
between versions of GTFS files for a region?

Adding a ref:gtfs tag would not be very hard to do, but I would prefer a
scheme with ref:OPERATOR, because some stops may be served by multiple
operators and thus be present in multiple GTFS feeds.

Polyglot

2018-01-16 21:07 GMT+01:00 Stephen Sprunk :

> On 2018-01-16 08:30, Mike N wrote:
>
>> On 1/16/2018 7:37 AM, Yash Ganthe wrote:
>>
>>> What caught my attention is a project called Go Sync
>>> https://wiki.openstreetmap.org/wiki/GO-Sync
>>> The description indicates that it is for syncing GTFS with OSM. But I am
>>> not sure what type of account is needed for that.
>>>
>>
>>
>>  From what I recall, Go-Sync only synchronizes stops but not the GTFS
>> route paths with OSM route relations.
>>
>>  The routing samples on the OpenStreetMap web site will not
>> automatically give public transport trip routing because the full GTFS
>> schedule information is not in OSM.   It would require a separate
>> local site such as OpenTripPlanner which automatically combines full
>> GTFS with OSM data.   There are some companies providing large scale
>> instances of OpenTripPlanner which may cover your area.
>>
>
> What I'd like to see is some sort of tag on OSM objects (stops, routes,
> etc.) listing the GTFS ID numbers so that tools can more easily connect the
> two; that should be easy enough if someone defines a scheme and gets the
> few relevant tools to use it.
>
> The lat/long for stops in GTFS data is often questionable, so it would be
> good to have some way for folks to be able to fix the stop locations in OSM
> and not get overwritten by another import later.  In many places
> (especially in the US), GTFS data will change every few months, so if we
> want it in OSM at all, we need a scheme that can deal with regular bulk
> imports without losing quality where local OSM editors have improved things
> by hand.
>
> S
>
> --
> Stephen Sprunk  "Those people who think they know everything
> CCIE #3723 are a great annoyance to those of us who do."
> K5SSS --Isaac Asimov
>
>
> ___
> Talk-transit mailing list
> Talk-transit@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-transit
>
___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit

Re: [Talk-transit] Uploading public transport data on OSM

[Yash Ganthe]

What caught my attention is a project called Go Sync
https://wiki.openstreetmap.org/wiki/GO-Sync
The description indicates that it is for syncing GTFS with OSM. But I am
not sure what type of account is needed for that.

Yes, the permissions from the agency will be obtained.

On Tue, Jan 16, 2018 at 5:47 PM, Jo  wrote:

> Hi, the very first question that comes to mind is: is it under a suitable
> license, or can you get explicit permission from the operator to contribute
> it to OpenStreetMap?
>
> The next issue is that there is no direct way to add GTFS to OSM.
> stops.txt can be converted to nodes for the stops, but the other files need
> to be processed to generate route relations containing lists of these stops.
>
> Sometimes there are shape files in GTFS, if that permission is granted
> these can be used as guides to know where the buses pass, but converting
> them to something useful in OSM (route and route_master) relations is very
> involved.
>
> I'm proposing a project for GSoC (Google Summer of Code) to create a web
> interface to help with such conversion and followup on its progress, so if
> you know students who know Python, Django and PostGIS, let them contact me.
>
> Polyglot
>
> 2018-01-16 12:44 GMT+01:00 Yash Ganthe :
>
>> Hi,
>>
>> If I have a GTFS data of my agency, can I upload it to OSM? Do I need to
>> convert it to any other format before uploading? Do I need to create a
>> special account in OSM to be able to upload the data? Will it start showing
>> the public transport options on the map similar to Google Maps?
>>
>> Regards,
>> Yash
>>
>>
>> ___
>> Talk-transit mailing list
>> Talk-transit@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-transit
>>
>>
>
> ___
> Talk-transit mailing list
> Talk-transit@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-transit
>
>
___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit

Re: [Talk-transit] Uploading public transport data on OSM

[Jo]

Hi, the very first question that comes to mind is: is it under a suitable
license, or can you get explicit permission from the operator to contribute
it to OpenStreetMap?

The next issue is that there is no direct way to add GTFS to OSM. stops.txt
can be converted to nodes for the stops, but the other files need to be
processed to generate route relations containing lists of these stops.

Sometimes there are shape files in GTFS, if that permission is granted
these can be used as guides to know where the buses pass, but converting
them to something useful in OSM (route and route_master) relations is very
involved.

I'm proposing a project for GSoC (Google Summer of Code) to create a web
interface to help with such conversion and followup on its progress, so if
you know students who know Python, Django and PostGIS, let them contact me.

Polyglot

2018-01-16 12:44 GMT+01:00 Yash Ganthe :

> Hi,
>
> If I have a GTFS data of my agency, can I upload it to OSM? Do I need to
> convert it to any other format before uploading? Do I need to create a
> special account in OSM to be able to upload the data? Will it start showing
> the public transport options on the map similar to Google Maps?
>
> Regards,
> Yash
>
>
> ___
> Talk-transit mailing list
> Talk-transit@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-transit
>
>
___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit

[Talk-transit] Uploading public transport data on OSM

[Yash Ganthe]

Hi,

If I have a GTFS data of my agency, can I upload it to OSM? Do I need to
convert it to any other format before uploading? Do I need to create a
special account in OSM to be able to upload the data? Will it start showing
the public transport options on the map similar to Google Maps?

Regards,
Yash
___
Talk-transit mailing list
Talk-transit@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-transit