Hi, We might be able to get some data from Train On Map site of Indian Railways?
https://enquiry.indianrail.gov.in/ntes/trainOnMap.jsp (good to see the map is working again.. last few months it had been down after google api changes) The browser console is yielding quite some stuff. One JS file there is holding some data: https://enquiry.indianrail.gov.in/ntes/js/stnCodesWithNamesArrayStr.js (note: you may see weird chars.. that's unicode hindi, save the file locally and you'll see the chars coming proper. ) ---------- Another dataset we could get from here: Train (routes) data (unique code and name), with names in Hindi too. See this API URL, I tried with wget and am getting results without having to do any cookie sessions etc: https://enquiry.indianrail.gov.in/ntes/SearchTrain?trainNo=22 Change last arg for different results.. It gives max 30 results. I changed the output to CSV by this process; it can be scripted: 1. Ran though https://www.freeformatter.com/json-formatter.html which put quotes around all the keys. (But one could script this too?) 2. There is a "function(){..." line which needs to be taken care of. 3. In advanced text editor, did Find+Replace-All on the following terms: [search string] >> [replace with] function(){return _LANG==="en-us"?" >> " ":" >> ", "trainNameHindi": " "}, >> ", (don't remove quotes) 4. Now it becomes valid JSON. OpenRefine recognizes it and converts it to tabular form, and even this site converts to CSV: https://konklone.io/json/ Regards Nikhil VJ Pune, India On Friday, January 4, 2019 at 9:22:12 PM UTC+5:30, Srihari Thalla wrote: > > Thanks for the mention Arun! > > I have now updated the crawler - removing tabs, unwanted newlines, leading > and trailing spaces in the data columns. > > Here is the latest links to download: > JSON: > https://api.apify.com/v1/execs/7t9roKQ9yp6T8ZnpR/results?format=json&simplified=1&attachment=1 > CSV: > https://api.apify.com/v1/execs/7t9roKQ9yp6T8ZnpR/results?format=csv&simplified=1&attachment=1 > > Hope this helps! > > @Jasvinder I think one solution to extract locations for the stations is > via Overpass using the station codes and combining them to the spreadsheet. > > -- Srihari > > > On Fri, 4 Jan 2019 at 12:43, Jasvinder Singh <[email protected] > <javascript:>> wrote: > >> Dear Arun, >> Exactly the type of simple data sheet that newbies can understand. >> However how location (Coordinates) is linked in this file? >> >> On Fri, Jan 4, 2019 at 12:11 PM Arun Ganesh <[email protected] >> <javascript:>> wrote: >> >>> Spreadsheet if anyone wants to explore: >>> https://docs.google.com/spreadsheets/d/1AFwl_5cB9qD39VWNox1LoeL3tGaGB22f7p4vc7IyMqY/edit#gid=0 >>> >>> There are 16,770 station entries of which 11,660 seem to be currently >>> operational according to the expiry date of 2999. >>> >>> Filtering out goods stations, there are 9835 entries. This still seems >>> to include a few yards and cabins that are not legitimate stations. Also >>> noticed quite a few spelling and formatting issues in the names. The >>> station codes look correct. Some amount of manual cleanup is needed on this >>> list. >>> >>> The official number of stations according to IR is 7349 stations (as of >>> 2017) >>> <http://www.indianrailways.gov.in/railwayboard/uploads/directorate/stat_econ/IRSP_2016-17/Facts_Figure/Fact_Figures%20English%202016-17.pdf> >>> and 1817 halts/block huts (2013) >>> <http://www.indianrailways.gov.in/railwayboard/uploads/directorate/stat_econ/downloads/Data_Bank.pdf>. >>> >>> >>> >>> On Fri, Jan 4, 2019 at 11:53 AM Arun Ganesh <[email protected] >>> <javascript:>> wrote: >>> >>>> Beauty of the internet, crawler got done by Srihari: >>>> https://twitter.com/sriharithalla/status/1080801313707896837 >>>> >>>> JSON data: >>>> https://api.apify.com/v1/execs/TsBwnYutP5u9FCKp5/results?format=json&simplified=1 >>>> >>>> I'm in the process of doing a little bit of cleanup using openrefine >>>> and will share on a spreadsheet. >>>> >>>> On Fri, Jan 4, 2019 at 10:01 AM Jasvinder Singh <[email protected] >>>> <javascript:>> wrote: >>>> >>>>> Dear All, >>>>> >>>>> Not all the members are familiar with intricacies of the data >>>>> collection for such projects. Since this seems to be a crowd sourcing >>>>> endeavour, I suggest that the basic data collection protocol be >>>>> enumerated >>>>> for newbies so that they can also contribute data which can then be put >>>>> in >>>>> proper format by professionals. >>>>> >>>>> Regards, >>>>> >>>>> Jasvinder Singh >>>>> >>>>> On Tue, Dec 25, 2018 at 10:32 AM Nikhil VJ <[email protected] >>>>> <javascript:>> wrote: >>>>> >>>>>> Hi folks, >>>>>> >>>>>> There's a project afoot in the OpenStreetMap and Wikidata communities >>>>>> to get together Indian Railways data. >>>>>> >>>>>> One major part of it: Properly mapping all the railway stations of >>>>>> India, and ensuring they have wikidata entries. >>>>>> >>>>>> Here's a wiki page set up for it: >>>>>> https://www.wikidata.org/wiki/Wikidata:WikiProject_Indian_Railways >>>>>> >>>>>> I'm cross-posting from OpenStreetMap India Telegram >>>>>> <https://t.me/OSMIndia> group: >>>>>> >>>>>> (Arun Ganesh): There seems to be around 7000 stations located. There >>>>>> still ~1.5k missing. A lot more need names, refs and wikidata links. >>>>>> Overpass: *http://overpass-turbo.eu/s/EC4 >>>>>> <http://overpass-turbo.eu/s/EC4>* >>>>>> >>>>>> >>>>>> (Srihari Thalla) : Last year I created two MapRoulette Challenges to >>>>>> tag station codes and add Wiki tags >>>>>> *https://maproulette.org/mr3/challenge/2403 >>>>>> <https://maproulette.org/mr3/challenge/2403>* >>>>>> *https://maproulette.org/mr3/challenge/2404 >>>>>> <https://maproulette.org/mr3/challenge/2404>* >>>>>> >>>>>> -------- >>>>>> >>>>>> The overpass query above queries the whole country and may be slow or >>>>>> timeout. I adapted the query to work only on the map area being >>>>>> viewed, so you can zoom into smaller regions. And changed a few things, >>>>>> included a legend in the comments to explain. >>>>>> >>>>>> *https://overpass-turbo.eu/s/EL9 <https://overpass-turbo.eu/s/EL9>* >>>>>> >>>>>> >>>>>> Want to get involved? Engage here. >>>>>> >>>>>> >>>>>> >>>>>> Regards >>>>>> >>>>>> Nikhil VJ >>>>>> >>>>>> Pune, India >>>>>> >>>>>> >>>>>> PS: Posting on an older thread from '15 that had the perfect subject >>>>>> line - didn't want to create yet another new thread. Pro tip: Use >>>>>> Datameet from >>>>>> google groups <https://groups.google.com/forum/#!forum/datameet> - >>>>>> its more fun and you can find stuff that was posted long before you >>>>>> joined. >>>>>> >>>>>> -- >>>>>> Datameet is a community of Data Science enthusiasts in India. Know >>>>>> more about us by visiting http://datameet.org >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "datameet" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected] <javascript:>. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> Datameet is a community of Data Science enthusiasts in India. Know >>>>> more about us by visiting http://datameet.org >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "datameet" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected] <javascript:>. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>> Datameet is a community of Data Science enthusiasts in India. Know more >>> about us by visiting http://datameet.org >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "datameet" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> Datameet is a community of Data Science enthusiasts in India. Know more >> about us by visiting http://datameet.org >> --- >> You received this message because you are subscribed to the Google Groups >> "datameet" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups "datameet" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
