Hi Folks, I caught up with this project over the last week, revamped my coding and methods and now have a workflow in place where:
1. The GTFS realtime VehiclePositions data of buses released by Open Transit Data Delhi is being archived once every minute 2. A clearing-house system processes the data files at regular intervals and appends it to a CSV 3. Duplications and bad data are being detected and avoided 4. Summary data with per-file stats is being logged to accompany the raw data collecting 5. Subsets of the summary data are being made for visualisation 6. Two time-series data visualisations are now up showing the metrics of the realtime data, and the latency. 7. The full data collecting since 28 April is compressed into a .7z archive every morning and posted online for downloading. You'll find everything here: https://server.nikhilvj.co.in/delhirt/ Also, just to repeat a link shared earlier, visualisation of current vehicle locations on a map: http://server.nikhilvj.co.in:8080/ The .7z has full raw data - inviting you to dig into it. Cheers Nikhil VJ, Pune, India On Sunday, April 28, 2019 at 11:30:45 PM UTC+5:30, Nikhil VJ wrote: > > Hi Folks, > > Early this month the real-time GTFS feed of Delhi transit had gotten > inflated with duplicating/redundant entries and buggy - They announced a > downtime <https://otd.delhi.gov.in/announcements/> to make fixes. > > Maybe it's the elections code-of-conduct or some other to-do work for > which announcement has not been made, but I checked again today, and the > Delhi realtime GTFS is now awesome! Clean and crisp feed, no repetitions, > very good latency, lat-longs seem accurate, clear route and trip ids given. > > I've deployed the live feed on a map visualization called OneBusAway > visualizer > <https://github.com/OneBusAway/onebusaway-gtfs-realtime-visualizer/>: > http://server.nikhilvj.co.in:8080/ > > I also downloaded their static GTFS data zip, and it's visualized here > using Transitfeed's ScheduleViewer <https://github.com/google/transitfeed> > : > http://server.nikhilvj.co.in:5060/ > > > I'm not getting too much into analysing these datasets right now, but am > re-starting long-term archival of the realtime feed which I had > discontinued last month owing to the aforementioned issues. > > > Note: The viz programs behind these tend to crash after prolonged uptime > (hey I just use 'em.. this is some pretty complicated programming so I'm > glad they work and are open-source!), so I've set them to restart every > hour, and they take a couple of mins to rev up. If you find the link not > working, please check back in 5-10 mins and if it's still down then let me > know. You can email me on nikhil.js [at] gmail.com . > > > Regards > Nikhil VJ > Pune, India > > See full thread on google groups > <https://groups.google.com/d/topic/datameet/exEoW8aGHu0/discussion> > -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups "datameet" group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/68da2ac2-ed7e-49a3-9a9c-ac8aaf837876%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
