Re: [datameet] Open Transit Data: Delhi

[Pravesh Biyani]

Hi DevDatta,  so sorry.

It will be activated tomorrow.

On Sat, Oct 22, 2022, 12:47 Devdatta Tengshe  wrote:

> Hi Pravesh,
> I have signed up for the API access to live data, but haven't heard back
> on when it would be activiated. Would you be able to tell us how long this
> activation step usually takes?
>
> I couldn't figure out who to ask, so am asking it here.
>
>
> Regards,
> Devdatta
>
>
> On Fri, 30 Sept 2022 at 02:07, Pravesh Biyani  wrote:
>
>> Dear All
>>
>> I am Pravesh Biyani. I am an associate professor at IIIT Delhi and head
>> the Centre for Sustainable Mobility too.  We have been working with the
>> Delhi govt and specifically the transport department here since last many
>> years.  One of the outputs of our work is the Open Transit Data for around
>> 3000 buses of Delhi.  (https://otd.delhi.gov.in).
>>
>> However, there are more than 7000 buses in Delhi.
>>
>> We have created the static data (that is reasonably accurate now) and
>> have been working with the vendor that installed GPS to collect the
>> location data and putting the "routes" to these data points in real-time to
>> form a complete data of buses.  Our system is now working and
>> reasonably stable. However, not fully stable.
>>
>> We would like to engage with the community to take this project forward,
>> so that project runs successfully forever and in a community driven sense.
>>
>> We also have funding to enable this.
>>
>> I am looking forward to hearing from you on this and will be happy to
>> demo and show case the stuff we have built.
>>
>> Let me know your thoughts, and if needed, we can have a joint
>> webinar/call to discuss the same.
>>
>> Thanks
>> Pravesh
>>
>>
>> On Sat, Sep 24, 2022 at 12:36 PM Shivangi Desai 
>> wrote:
>>
>>> Hey All,
>>>
>>> Have any of you had the need to show the boundary data (state, district,
>>> block, village etc) in regional language?
>>> If so how did you go about it?
>>> Did you need to translate it and store it in database or is there an
>>> android hack or some other way to show translated data on a phone
>>> application?
>>>
>>>
>>> Thanks & Regards,
>>> Shivangi
>>>
>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>> about us by visiting http://datameet.org
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to datameet+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/datameet/82B316FE-5783-4259-AAF0-028F9C934A13%40gmail.com
>>> .
>>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CAG3fi50f2SVcGvgm%3DW%3DPPYU6vMLYx8E412%3DiO8rbyH6f0ovWLg%40mail.gmail.com
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAG3fi50R2A%2BusMzBnNbW_mV1MQMnsgC%3D2z6eovRFfrOVbcF2cg%40mail.gmail.com.

Re: [datameet] Open Transit Data: Delhi

[Devdatta Tengshe]

Hi Pravesh,
I have signed up for the API access to live data, but haven't heard back on
when it would be activiated. Would you be able to tell us how long this
activation step usually takes?

I couldn't figure out who to ask, so am asking it here.


Regards,
Devdatta


On Fri, 30 Sept 2022 at 02:07, Pravesh Biyani  wrote:

> Dear All
>
> I am Pravesh Biyani. I am an associate professor at IIIT Delhi and head
> the Centre for Sustainable Mobility too.  We have been working with the
> Delhi govt and specifically the transport department here since last many
> years.  One of the outputs of our work is the Open Transit Data for around
> 3000 buses of Delhi.  (https://otd.delhi.gov.in).
>
> However, there are more than 7000 buses in Delhi.
>
> We have created the static data (that is reasonably accurate now) and have
> been working with the vendor that installed GPS to collect the location
> data and putting the "routes" to these data points in real-time to form a
> complete data of buses.  Our system is now working and reasonably stable.
> However, not fully stable.
>
> We would like to engage with the community to take this project forward,
> so that project runs successfully forever and in a community driven sense.
>
> We also have funding to enable this.
>
> I am looking forward to hearing from you on this and will be happy to demo
> and show case the stuff we have built.
>
> Let me know your thoughts, and if needed, we can have a joint webinar/call
> to discuss the same.
>
> Thanks
> Pravesh
>
>
> On Sat, Sep 24, 2022 at 12:36 PM Shivangi Desai 
> wrote:
>
>> Hey All,
>>
>> Have any of you had the need to show the boundary data (state, district,
>> block, village etc) in regional language?
>> If so how did you go about it?
>> Did you need to translate it and store it in database or is there an
>> android hack or some other way to show translated data on a phone
>> application?
>>
>>
>> Thanks & Regards,
>> Shivangi
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/82B316FE-5783-4259-AAF0-028F9C934A13%40gmail.com
>> .
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAG3fi50f2SVcGvgm%3DW%3DPPYU6vMLYx8E412%3DiO8rbyH6f0ovWLg%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAOQOUMsyGTiw%2Ban5c0grOTLNj4H25-zpE0L4hjYAitnVTthFJA%40mail.gmail.com.

[datameet] Open Transit Data: Delhi

[Pravesh Biyani]

Dear All

I am Pravesh Biyani. I am an associate professor at IIIT Delhi and head the
Centre for Sustainable Mobility too.  We have been working with the Delhi
govt and specifically the transport department here since last many years.
One of the outputs of our work is the Open Transit Data for around 3000
buses of Delhi.  (https://otd.delhi.gov.in).

However, there are more than 7000 buses in Delhi.

We have created the static data (that is reasonably accurate now) and have
been working with the vendor that installed GPS to collect the location
data and putting the "routes" to these data points in real-time to form a
complete data of buses.  Our system is now working and reasonably stable.
However, not fully stable.

We would like to engage with the community to take this project forward, so
that project runs successfully forever and in a community driven sense.

We also have funding to enable this.

I am looking forward to hearing from you on this and will be happy to demo
and show case the stuff we have built.

Let me know your thoughts, and if needed, we can have a joint webinar/call
to discuss the same.

Thanks
Pravesh


On Sat, Sep 24, 2022 at 12:36 PM Shivangi Desai 
wrote:

> Hey All,
>
> Have any of you had the need to show the boundary data (state, district,
> block, village etc) in regional language?
> If so how did you go about it?
> Did you need to translate it and store it in database or is there an
> android hack or some other way to show translated data on a phone
> application?
>
>
> Thanks & Regards,
> Shivangi
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/82B316FE-5783-4259-AAF0-028F9C934A13%40gmail.com
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAG3fi50f2SVcGvgm%3DW%3DPPYU6vMLYx8E412%3DiO8rbyH6f0ovWLg%40mail.gmail.com.

Re: [datameet] Open Transit Data - Delhi

[Guneet Narula]

Amazing! Thanks Nikhil. Just downloaded the raw data, will explore it 

On Friday, 21 June 2019 15:21:05 UTC+5:30, Nikhil VJ wrote:
>
> Hi Folks,
>
> I caught up with this project over the last week, revamped my coding and 
> methods and now have a workflow in place where:
>
> 1. The GTFS realtime VehiclePositions data of buses released by Open 
> Transit Data Delhi is being archived once every minute
> 2. A clearing-house system processes the data files at regular intervals 
> and appends it to a CSV
> 3. Duplications and bad data are being detected and avoided
> 4. Summary data with per-file stats is being logged to accompany the raw 
> data collecting
> 5. Subsets of the summary data are being made for visualisation
> 6. Two time-series data visualisations are now up showing the metrics of 
> the realtime data, and the latency.
> 7. The full data collecting since 28 April is compressed into a .7z 
> archive every morning and posted online for downloading.
>
> You'll find everything here:
> https://server.nikhilvj.co.in/delhirt/
>
> Also, just to repeat a link shared earlier, visualisation of current 
> vehicle locations on a map: http://server.nikhilvj.co.in:8080/
>
> The .7z has full raw data - inviting you to dig into it.
>
>
> Cheers
> Nikhil VJ, Pune, India
>
>
> On Sunday, April 28, 2019 at 11:30:45 PM UTC+5:30, Nikhil VJ wrote:
>>
>> Hi Folks,
>>
>> Early this month the real-time GTFS feed of Delhi transit had gotten 
>> inflated with duplicating/redundant entries and buggy - They announced a 
>> downtime  to make fixes. 
>>
>> Maybe it's the elections code-of-conduct or some other to-do work for 
>> which announcement has not been made, but I checked again today, and the 
>> Delhi realtime GTFS is now awesome! Clean and crisp feed, no repetitions, 
>> very good latency, lat-longs seem accurate, clear route and trip ids given.
>>
>> I've deployed the live feed on a map visualization called OneBusAway 
>> visualizer 
>> : 
>> http://server.nikhilvj.co.in:8080/
>>
>> I also downloaded their static GTFS data zip, and it's visualized here 
>> using Transitfeed's ScheduleViewer 
>> : 
>> http://server.nikhilvj.co.in:5060/
>>
>>
>> I'm not getting too much into analysing these datasets right now, but am 
>> re-starting long-term archival of the realtime feed which I had 
>> discontinued last month owing to the aforementioned issues.
>>
>>
>> Note: The viz programs behind these tend to crash after prolonged uptime 
>> (hey I just use 'em.. this is some pretty complicated programming so I'm 
>> glad they work and are open-source!), so I've set them to restart every 
>> hour, and they take a couple of mins to rev up. If you find the link not 
>> working, please check back in 5-10 mins and if it's still down then let me 
>> know. You can email me on nikhil.js [at] gmail.com .
>>
>>
>> Regards
>> Nikhil VJ
>> Pune, India
>>
>> See full thread on google groups 
>> 
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/d6e3ae19-ce0f-4125-9062-2c98dfa61ef9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Open Transit Data - Delhi

[Nikhil VJ]

Hi Folks,

I caught up with this project over the last week, revamped my coding and 
methods and now have a workflow in place where:

1. The GTFS realtime VehiclePositions data of buses released by Open 
Transit Data Delhi is being archived once every minute
2. A clearing-house system processes the data files at regular intervals 
and appends it to a CSV
3. Duplications and bad data are being detected and avoided
4. Summary data with per-file stats is being logged to accompany the raw 
data collecting
5. Subsets of the summary data are being made for visualisation
6. Two time-series data visualisations are now up showing the metrics of 
the realtime data, and the latency.
7. The full data collecting since 28 April is compressed into a .7z archive 
every morning and posted online for downloading.

You'll find everything here:
https://server.nikhilvj.co.in/delhirt/

Also, just to repeat a link shared earlier, visualisation of current 
vehicle locations on a map: http://server.nikhilvj.co.in:8080/

The .7z has full raw data - inviting you to dig into it.


Cheers
Nikhil VJ, Pune, India


On Sunday, April 28, 2019 at 11:30:45 PM UTC+5:30, Nikhil VJ wrote:
>
> Hi Folks,
>
> Early this month the real-time GTFS feed of Delhi transit had gotten 
> inflated with duplicating/redundant entries and buggy - They announced a 
> downtime  to make fixes. 
>
> Maybe it's the elections code-of-conduct or some other to-do work for 
> which announcement has not been made, but I checked again today, and the 
> Delhi realtime GTFS is now awesome! Clean and crisp feed, no repetitions, 
> very good latency, lat-longs seem accurate, clear route and trip ids given.
>
> I've deployed the live feed on a map visualization called OneBusAway 
> visualizer 
> : 
> http://server.nikhilvj.co.in:8080/
>
> I also downloaded their static GTFS data zip, and it's visualized here 
> using Transitfeed's ScheduleViewer 
> : 
> http://server.nikhilvj.co.in:5060/
>
>
> I'm not getting too much into analysing these datasets right now, but am 
> re-starting long-term archival of the realtime feed which I had 
> discontinued last month owing to the aforementioned issues.
>
>
> Note: The viz programs behind these tend to crash after prolonged uptime 
> (hey I just use 'em.. this is some pretty complicated programming so I'm 
> glad they work and are open-source!), so I've set them to restart every 
> hour, and they take a couple of mins to rev up. If you find the link not 
> working, please check back in 5-10 mins and if it's still down then let me 
> know. You can email me on nikhil.js [at] gmail.com .
>
>
> Regards
> Nikhil VJ
> Pune, India
>
> See full thread on google groups 
> 
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/68da2ac2-ed7e-49a3-9a9c-ac8aaf837876%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Open Transit Data - Delhi

[Nikhil VJ]

Hi Folks,

Early this month the real-time GTFS feed of Delhi transit had gotten 
inflated with duplicating/redundant entries and buggy - They announced a 
downtime  to make fixes. 

Maybe it's the elections code-of-conduct or some other to-do work for which 
announcement has not been made, but I checked again today, and the Delhi 
realtime GTFS is now awesome! Clean and crisp feed, no repetitions, very 
good latency, lat-longs seem accurate, clear route and trip ids given.

I've deployed the live feed on a map visualization called OneBusAway 
visualizer 
: 
http://server.nikhilvj.co.in:8080/

I also downloaded their static GTFS data zip, and it's visualized here 
using Transitfeed's ScheduleViewer : 
http://server.nikhilvj.co.in:5060/


I'm not getting too much into analysing these datasets right now, but am 
re-starting long-term archival of the realtime feed which I had 
discontinued last month owing to the aforementioned issues.


Note: The viz programs behind these tend to crash after prolonged uptime 
(hey I just use 'em.. this is some pretty complicated programming so I'm 
glad they work and are open-source!), so I've set them to restart every 
hour, and they take a couple of mins to rev up. If you find the link not 
working, please check back in 5-10 mins and if it's still down then let me 
know. You can email me on nikhil.js [at] gmail.com .


Regards
Nikhil VJ
Pune, India

See full thread on google groups 


-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Open Transit Data - Delhi

[Nikhil VJ]

Hi Devdatta,

Thanks for asking, happy to provide the documentation here:

*file* : name of the GTFS-RT feed file that was processed (there's one
every 30 secs, so named by epoch timestamp of downloading)
*num_vehicles* : number of vehicle locations (entities) in the feed
*feed_timestamp* : timestamp of the feed that's there in the header. If you
got the same value as the last feed, it's duplicate. (Whoops I just found
out there were some 13k dupes in my data!)
*feed_time* : human, IST form of timestamp
*incrementality* : that's another field in the feed's header
*bad_count* : number of entities that had missing or flawed lat-longs like
0,0
*earliest* : timestamp of the earliest (farthest back in time) vehicle
entity in the file (apart from feed timestamp, each vehicle location
'entity' must carry a timestamp of its own, because we can't assume all
700-odd vehicles sent in their lat-longs at exactly the same time. We're
doing asynchronous business here.)
*diff1* : seconds gap between earliest and feed_timestamp
*latest* : time of latest entity
*diff2* : seconds gap between latest and feed_timestamp

What diff1, diff2 tells me : How "dated" is the information in the feed;
and how "recent" is the information in it. Supposing diff2 was too large
consistently, then I wouldn't bother to download a fresh feed every 30 secs
which is the standard minimum refreshing time as per GTFS-RT specs.

For reference, Sample json output from a feed:

{'header': {'gtfsRealtimeVersion': '2.0',
  'incrementality': 'FULL_DATASET',
  'timestamp': '1550596818'},
 'entity': [{'id': 'vehicle',
   'vehicle': {'trip': {'tripId': '6255', 'routeId': '225'},
'position': {'latitude': 28.610946655273438,
 'longitude': 76.980224609375,
 'speed': 0.0},
'timestamp': '1550596773',
'vehicle': {'id': 'DL1PD0716', 'label': 'DL1PD0716'}}},

Repeat-link to my code example:
http://kyso.io/answerquest/delhi-gtfs-rt-feed-file-analysis

--
Cheers,
Nikhil VJ, Pune, India
http://nikhilvj.co.in


On Fri, Mar 22, 2019 at 9:53 AM Devdatta Tengshe 
wrote:

> Hi Nikhil,
>
> Thanks for sharing this data.
> I had a question about the 'delhi_vehicle_reports.csv' file.
>
> If there any documentation about the fields in this file?
>
> I see the following headers:
>
>>
>> file,num_vehicles,feed_timestamp,feed_time,incrementality,bad_count,earliest,diff1,latest,diff2
>>
> & I'm wondering what they are
>
> Regards,
> Devdatta
>
>
> On Thu, Mar 21, 2019 at 1:24 PM Nikhil VJ  wrote:
>
>> Hi folks,
>>
>> I have been archiving Delhi's bus realtime gtfs feeds on my server since
>> a month now and collating the data into a flat CSV. Sharing that data at
>> this link for download and analysis:
>> https://server.nikhilvj.co.in/place1/
>>
>> Hoping some folks can make some analysis, visualizations or so of it - I
>> don't have time to delve too much into that right now. It's been a great
>> learning experience arranging the scripts and structures on my digitalocean
>> server to make this long-term continuous archival process possible.
>>
>> Disclaimer: Default reply to every sage advice that starts with "Why
>> don't you.." is : "Sounds good, please do it and get back with the
>> results." I'm satisfied at my end and am sharing the data wealth here for
>> others to take forward, so don't bug me, just take it and go! ;)
>>
>>
>> Some more notes:
>> 1. Get 7zip portable / p7zipfull to uncompress it. The uncompressed one
>> is around 6 gigs, compressed its about half a gig.
>> 2. There may be many repetitions in the data though, since the feeds were
>> coming in every 30 secs, plus from my last email the analysis showed there
>> were repetitions in one feed itself. So it's a data cleaning challenge for
>> you here to remove repetitions. (Do it - don't expect things to be already
>> done for you unless you're paying a fortune for it!)
>> 3. If there is too much traffic on my server then I'll lock it all down
>> with username-password restrictions. So don't do silly things like telling
>> a whole class of students to download it from online only. Use a pen drive
>> or your LAN.
>> 4. There is an accompanying reports csv that gives file-level summaries.
>> 5. Timestamps are in epoch format in UTC timezone (as per GTFS-RT specs).
>> Lookup "epoch converter". In the reports file i've added 5.5hrs to get
>> human times in IST.
>> 6. The data inside is covering all the dates from 19 Feb onwards. Moving
>> forward I might make the scripts store things month-wise or week-wise. Here
>> it was important to start asap.
>> 7. Every early morning my scripts will place a fresh version of the data
>> there and remove the previous day's one. So don't be downloading stuff from
>> there at 5am.
>> 8. Tip : Python? Wanna map? Check out folium.
>> 9. Tip : Folium? Wanna share the ipynb notebook? Check out kyso.io
>>
>> PS: Thanks JohnsonC for the kind words. But that is because I use
>> datameet from google groups instead of from my mailbox, unless it's an
>> immediate followup. 

Re: [datameet] Open Transit Data - Delhi

[Devdatta Tengshe]

Hi Nikhil,

Thanks for sharing this data.
I had a question about the 'delhi_vehicle_reports.csv' file.

If there any documentation about the fields in this file?

I see the following headers:

>
> file,num_vehicles,feed_timestamp,feed_time,incrementality,bad_count,earliest,diff1,latest,diff2
>
& I'm wondering what they are

Regards,
Devdatta


On Thu, Mar 21, 2019 at 1:24 PM Nikhil VJ  wrote:

> Hi folks,
>
> I have been archiving Delhi's bus realtime gtfs feeds on my server since a
> month now and collating the data into a flat CSV. Sharing that data at this
> link for download and analysis: https://server.nikhilvj.co.in/place1/
>
> Hoping some folks can make some analysis, visualizations or so of it - I
> don't have time to delve too much into that right now. It's been a great
> learning experience arranging the scripts and structures on my digitalocean
> server to make this long-term continuous archival process possible.
>
> Disclaimer: Default reply to every sage advice that starts with "Why don't
> you.." is : "Sounds good, please do it and get back with the results." I'm
> satisfied at my end and am sharing the data wealth here for others to take
> forward, so don't bug me, just take it and go! ;)
>
>
> Some more notes:
> 1. Get 7zip portable / p7zipfull to uncompress it. The uncompressed one is
> around 6 gigs, compressed its about half a gig.
> 2. There may be many repetitions in the data though, since the feeds were
> coming in every 30 secs, plus from my last email the analysis showed there
> were repetitions in one feed itself. So it's a data cleaning challenge for
> you here to remove repetitions. (Do it - don't expect things to be already
> done for you unless you're paying a fortune for it!)
> 3. If there is too much traffic on my server then I'll lock it all down
> with username-password restrictions. So don't do silly things like telling
> a whole class of students to download it from online only. Use a pen drive
> or your LAN.
> 4. There is an accompanying reports csv that gives file-level summaries.
> 5. Timestamps are in epoch format in UTC timezone (as per GTFS-RT specs).
> Lookup "epoch converter". In the reports file i've added 5.5hrs to get
> human times in IST.
> 6. The data inside is covering all the dates from 19 Feb onwards. Moving
> forward I might make the scripts store things month-wise or week-wise. Here
> it was important to start asap.
> 7. Every early morning my scripts will place a fresh version of the data
> there and remove the previous day's one. So don't be downloading stuff from
> there at 5am.
> 8. Tip : Python? Wanna map? Check out folium.
> 9. Tip : Folium? Wanna share the ipynb notebook? Check out kyso.io
>
> PS: Thanks JohnsonC for the kind words. But that is because I use datameet
> from google groups instead of from my mailbox, unless it's an immediate
> followup. So it's like stackexchange for me, and it saves me time and
> effort.
>
> Cheers
> Nikhil VJ
> Pune, India
>
>
> On Thursday, March 14, 2019 at 11:50:24 AM UTC+5:30, JohnsonC wrote:
>>
>>
>> This is helpful.
>> Thanks for updating on this Nikhil.
>> This thread was from November and you bothered to search and update it.
>>
>> Thanks,
>>
>>
>> On Wed, 13 Mar 2019 at 20:38, Nikhil VJ  wrote:
>>
>>> Hi Folks,
>>>
>>> Sometime last month my API key for the realtime feed of Delhi bus data
>>> started working. (link to register for y
>>> ours).
>>>
>>> Here's an "unboxing" of one gtfs-realtime vehicleposition feed file from
>>> there:
>>> http://kyso.io/answerquest/delhi-gtfs-rt-feed-file-analysis
>>>
>>>
>>> Note: I'm guessing this is not DTC but other bus services operating in
>>> Delhi.
>>>
>>>
>>> - Nikhil VJ
>>> Pune, India
>>> https://nikhilvj.co.in
>>>
>>>
>>> On Thursday, November 29, 2018 at 5:17:39 PM UTC+5:30, Nikhil VJ wrote:

 Hi Arun,

 This data doesn't include any shapes.txt file, probably that script
 requires it. shapes.txt is not mandatory in GTFS. The routes are defined as
 sequence of stops in stop_times.txt (multiplied by the number of trips in a
 day that is).

 There's room here for improvement. Here's a full gtfs validator output
 for the delhi data :
 http://nikhilvj.co.in/files/delhi_gtfs/delhi-gtfs-2.html

 One peculiarity : The routes have been split up into separate onward
 and return journey routes.

 If anybody knows someone on the technical team of this, kindly connect
 me with them. The project leads are probably too busy with handling
 realtime data access requests and won't take too kindly to feedback about
 what improvements can be done on static side, but I might be able to put
 something across to the technical folks.

 You can zip up and import the static GTFS files on static GTFS Manager
  tool. If someone
 wants to draw the shapefiles of the routes and add them in, the "Default

Re: [datameet] Open Transit Data - Delhi

[Nikhil VJ]

Hi folks,

I have been archiving Delhi's bus realtime gtfs feeds on my server since a 
month now and collating the data into a flat CSV. Sharing that data at this 
link for download and analysis: https://server.nikhilvj.co.in/place1/ 

Hoping some folks can make some analysis, visualizations or so of it - I 
don't have time to delve too much into that right now. It's been a great 
learning experience arranging the scripts and structures on my digitalocean 
server to make this long-term continuous archival process possible.

Disclaimer: Default reply to every sage advice that starts with "Why don't 
you.." is : "Sounds good, please do it and get back with the results." I'm 
satisfied at my end and am sharing the data wealth here for others to take 
forward, so don't bug me, just take it and go! ;)


Some more notes:
1. Get 7zip portable / p7zipfull to uncompress it. The uncompressed one is 
around 6 gigs, compressed its about half a gig.
2. There may be many repetitions in the data though, since the feeds were 
coming in every 30 secs, plus from my last email the analysis showed there 
were repetitions in one feed itself. So it's a data cleaning challenge for 
you here to remove repetitions. (Do it - don't expect things to be already 
done for you unless you're paying a fortune for it!)
3. If there is too much traffic on my server then I'll lock it all down 
with username-password restrictions. So don't do silly things like telling 
a whole class of students to download it from online only. Use a pen drive 
or your LAN.
4. There is an accompanying reports csv that gives file-level summaries.
5. Timestamps are in epoch format in UTC timezone (as per GTFS-RT specs). 
Lookup "epoch converter". In the reports file i've added 5.5hrs to get 
human times in IST.
6. The data inside is covering all the dates from 19 Feb onwards. Moving 
forward I might make the scripts store things month-wise or week-wise. Here 
it was important to start asap.
7. Every early morning my scripts will place a fresh version of the data 
there and remove the previous day's one. So don't be downloading stuff from 
there at 5am.
8. Tip : Python? Wanna map? Check out folium.
9. Tip : Folium? Wanna share the ipynb notebook? Check out kyso.io

PS: Thanks JohnsonC for the kind words. But that is because I use datameet 
from google groups instead of from my mailbox, unless it's an immediate 
followup. So it's like stackexchange for me, and it saves me time and 
effort.

Cheers
Nikhil VJ
Pune, India


On Thursday, March 14, 2019 at 11:50:24 AM UTC+5:30, JohnsonC wrote:
>
>
> This is helpful. 
> Thanks for updating on this Nikhil.  
> This thread was from November and you bothered to search and update it.
>
> Thanks, 
>
>
> On Wed, 13 Mar 2019 at 20:38, Nikhil VJ > 
> wrote:
>
>> Hi Folks,
>>
>> Sometime last month my API key for the realtime feed of Delhi bus data 
>> started working. (link to register for y 
>> ours).
>>
>> Here's an "unboxing" of one gtfs-realtime vehicleposition feed file from 
>> there:
>> http://kyso.io/answerquest/delhi-gtfs-rt-feed-file-analysis
>>
>>
>> Note: I'm guessing this is not DTC but other bus services operating in 
>> Delhi.
>>
>>
>> - Nikhil VJ
>> Pune, India
>> https://nikhilvj.co.in
>>
>>
>> On Thursday, November 29, 2018 at 5:17:39 PM UTC+5:30, Nikhil VJ wrote:
>>>
>>> Hi Arun,
>>>
>>> This data doesn't include any shapes.txt file, probably that script 
>>> requires it. shapes.txt is not mandatory in GTFS. The routes are defined as 
>>> sequence of stops in stop_times.txt (multiplied by the number of trips in a 
>>> day that is).
>>>
>>> There's room here for improvement. Here's a full gtfs validator output 
>>> for the delhi data : 
>>> http://nikhilvj.co.in/files/delhi_gtfs/delhi-gtfs-2.html
>>>
>>> One peculiarity : The routes have been split up into separate onward and 
>>> return journey routes.
>>>
>>> If anybody knows someone on the technical team of this, kindly connect 
>>> me with them. The project leads are probably too busy with handling 
>>> realtime data access requests and won't take too kindly to feedback about 
>>> what improvements can be done on static side, but I might be able to put 
>>> something across to the technical folks.
>>>
>>> You can zip up and import the static GTFS files on static GTFS Manager 
>>>  tool. If someone 
>>> wants to draw the shapefiles of the routes and add them in, the "Default 
>>> Sequence" page will help you do that.
>>>
>>> --
>>> Cheers,
>>> Nikhil VJ
>>> +91-966-583-1250
>>> Pune, India
>>> http://nikhilvj.co.in
>>>
>>>
>>> On Tue, Nov 27, 2018 at 1:50 AM Arun Ganesh  wrote:
>>>
 Was anyone able to convert the GTFS feed into a geojson? 

 Tried https://github.com/BlinkTagInc/gtfs-to-geojson but for some 
 reason does not produce any route lines.

 -- 
 Datameet is a community of Data Science enthusiasts in India. Know more 
 about 

Re: [datameet] Open Transit Data - Delhi

[Johnson Chetty]

This is helpful.
Thanks for updating on this Nikhil.
This thread was from November and you bothered to search and update it.

Thanks,


On Wed, 13 Mar 2019 at 20:38, Nikhil VJ  wrote:

> Hi Folks,
>
> Sometime last month my API key for the realtime feed of Delhi bus data
> started working. (link to register for y
> ours).
>
> Here's an "unboxing" of one gtfs-realtime vehicleposition feed file from
> there:
> http://kyso.io/answerquest/delhi-gtfs-rt-feed-file-analysis
>
>
> Note: I'm guessing this is not DTC but other bus services operating in
> Delhi.
>
>
> - Nikhil VJ
> Pune, India
> https://nikhilvj.co.in
>
>
> On Thursday, November 29, 2018 at 5:17:39 PM UTC+5:30, Nikhil VJ wrote:
>>
>> Hi Arun,
>>
>> This data doesn't include any shapes.txt file, probably that script
>> requires it. shapes.txt is not mandatory in GTFS. The routes are defined as
>> sequence of stops in stop_times.txt (multiplied by the number of trips in a
>> day that is).
>>
>> There's room here for improvement. Here's a full gtfs validator output
>> for the delhi data :
>> http://nikhilvj.co.in/files/delhi_gtfs/delhi-gtfs-2.html
>>
>> One peculiarity : The routes have been split up into separate onward and
>> return journey routes.
>>
>> If anybody knows someone on the technical team of this, kindly connect me
>> with them. The project leads are probably too busy with handling realtime
>> data access requests and won't take too kindly to feedback about what
>> improvements can be done on static side, but I might be able to put
>> something across to the technical folks.
>>
>> You can zip up and import the static GTFS files on static GTFS Manager
>>  tool. If someone
>> wants to draw the shapefiles of the routes and add them in, the "Default
>> Sequence" page will help you do that.
>>
>> --
>> Cheers,
>> Nikhil VJ
>> +91-966-583-1250
>> Pune, India
>> http://nikhilvj.co.in
>>
>>
>> On Tue, Nov 27, 2018 at 1:50 AM Arun Ganesh  wrote:
>>
>>> Was anyone able to convert the GTFS feed into a geojson?
>>>
>>> Tried https://github.com/BlinkTagInc/gtfs-to-geojson but for some
>>> reason does not produce any route lines.
>>>
>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>> about us by visiting http://datameet.org
>>> ---
>>>
>>> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Warm Regards,
Johnson Chetty

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Open Transit Data - Delhi

[Nikhil VJ]

Hi Folks,

Sometime last month my API key for the realtime feed of Delhi bus data 
started working. (link to register for y 
ours).

Here's an "unboxing" of one gtfs-realtime vehicleposition feed file from 
there:
http://kyso.io/answerquest/delhi-gtfs-rt-feed-file-analysis


Note: I'm guessing this is not DTC but other bus services operating in 
Delhi.


- Nikhil VJ
Pune, India
https://nikhilvj.co.in


On Thursday, November 29, 2018 at 5:17:39 PM UTC+5:30, Nikhil VJ wrote:
>
> Hi Arun,
>
> This data doesn't include any shapes.txt file, probably that script 
> requires it. shapes.txt is not mandatory in GTFS. The routes are defined as 
> sequence of stops in stop_times.txt (multiplied by the number of trips in a 
> day that is).
>
> There's room here for improvement. Here's a full gtfs validator output for 
> the delhi data : http://nikhilvj.co.in/files/delhi_gtfs/delhi-gtfs-2.html
>
> One peculiarity : The routes have been split up into separate onward and 
> return journey routes.
>
> If anybody knows someone on the technical team of this, kindly connect me 
> with them. The project leads are probably too busy with handling realtime 
> data access requests and won't take too kindly to feedback about what 
> improvements can be done on static side, but I might be able to put 
> something across to the technical folks.
>
> You can zip up and import the static GTFS files on static GTFS Manager 
>  tool. If someone 
> wants to draw the shapefiles of the routes and add them in, the "Default 
> Sequence" page will help you do that.
>
> --
> Cheers,
> Nikhil VJ
> +91-966-583-1250
> Pune, India
> http://nikhilvj.co.in
>
>
> On Tue, Nov 27, 2018 at 1:50 AM Arun Ganesh  wrote:
>
>> Was anyone able to convert the GTFS feed into a geojson? 
>>
>> Tried https://github.com/BlinkTagInc/gtfs-to-geojson but for some reason 
>> does not produce any route lines.
>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>>
>>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Open Transit Data - Delhi

[Nikhil VJ]

Hi Arun,

This data doesn't include any shapes.txt file, probably that script
requires it. shapes.txt is not mandatory in GTFS. The routes are defined as
sequence of stops in stop_times.txt (multiplied by the number of trips in a
day that is).

There's room here for improvement. Here's a full gtfs validator output for
the delhi data : http://nikhilvj.co.in/files/delhi_gtfs/delhi-gtfs-2.html

One peculiarity : The routes have been split up into separate onward and
return journey routes.

If anybody knows someone on the technical team of this, kindly connect me
with them. The project leads are probably too busy with handling realtime
data access requests and won't take too kindly to feedback about what
improvements can be done on static side, but I might be able to put
something across to the technical folks.

You can zip up and import the static GTFS files on static GTFS Manager
 tool. If someone wants
to draw the shapefiles of the routes and add them in, the "Default
Sequence" page will help you do that.

--
Cheers,
Nikhil VJ
+91-966-583-1250
Pune, India
http://nikhilvj.co.in


On Tue, Nov 27, 2018 at 1:50 AM Arun Ganesh  wrote:

> Was anyone able to convert the GTFS feed into a geojson?
>
> Tried https://github.com/BlinkTagInc/gtfs-to-geojson but for some reason
> does not produce any route lines.
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[datameet] Open Transit Data - Delhi

[Yogada Joshi]

Hi,

Do you have any idea where I could get transit data for Bangalore ?

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[datameet] Open Transit Data - Delhi

[Sarath Guttikunda]

https://twitter.com/suraiya95/status/1065921782325145600
Open data platform for public transit in Delhi is live, with route and bus
shelter details and live GPS of cluster buses. Many thanks to @IIITDelhi
team led by @pravesh who made it for us! http://otd.delhi.gov.in  We will
add DTC buses when the new GPS vendor is in place.

--
*Dr. Sarath Guttikunda*

*http://www.urbanemissions.info *

*http://www.indiaairquality.info *

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.