Re: [datameet] GIS data of Mumbai related to COVID 19

2020-10-19 Thread Souvik Sankar Mitra
Hello Vijay, 
  Previously someone asked to scrape this site. I wrote a python script 
to download the shapefiles. You can check here - 
https://groups.google.com/g/datameet/c/tkQyV5FgeA0/m/J6VEhT_GAAAJ

On Tuesday, October 20, 2020 at 8:20:17 AM UTC+5:30 vjsap...@gmail.com 
wrote:

> Hello everyone,
>
> I am working on COVID project in M/E ward of Mumbai city. We are mapping 
> few municipal activities in GIS.
>
> Can anyone help me to get shape files from UDRI websites? Link - 
> http://www.loginmumbai.org/
>
> We want to use this data to identify existing facilities as well as 
> requirements for the future. 
>
> Thank you!
>
> On Wed, Sep 23, 2020 at 2:13 PM Souvik Sankar Mitra  
> wrote:
>
>> I scraped the site and got the data. But in some cases, Lat Long of 
>> Quarantine Centres are not correct in the site. 
>>
>> On Wed, Sep 23, 2020 at 9:39 AM Vijay Sapkal  wrote:
>>
>>> Hello Thejesh GN,
>>> Thank you for replying..!
>>>
>>> I have checked both links. I want to download data which is available on 
>>> view-source. Is it possible? If yes, could you please help me to do it. 
>>>
>>> Thank you,
>>> Vijay 
>>>
>>> On Tue, Sep 22, 2020 at 2:49 PM Thejesh GN  wrote:
>>>
>>>> https://bhugolgis.s3.ap-south-1.amazonaws.com/mcgm_covid19_map2.html
>>>>
>>>> Do the view source. All the map data is inside the view source
>>>>
>>>> view-source:
>>>> https://bhugolgis.s3.ap-south-1.amazonaws.com/mcgm_covid19_map2.html
>>>>
>>>>
>>>> Thej
>>>> --
>>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>> http://thejeshgn.com
>>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>>
>>>>
>>>> On Tue, 22 Sep 2020 at 14:44, Thejesh GN  wrote:
>>>>
>>>>> There is a PDF here. 
>>>>> https://stopcoronavirus.mcgm.gov.in/assets/docs/Containment-Zones.pdf
>>>>>
>>>>> Which link are you talking about?
>>>>>
>>>>> Thej
>>>>> --
>>>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>>> http://thejeshgn.com
>>>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>>>
>>>>>
>>>>> On Tue, 22 Sep 2020 at 10:57, Vijay Sapkal  wrote:
>>>>>
>>>>>> Hello everyone, 
>>>>>>
>>>>>> I am looking for GIS data of Mumbai related to COVID 19 like 
>>>>>> containment zones/areas, covid health centers/hospitals, quarantine 
>>>>>> centers 
>>>>>> (type 1 and 2). etc. marked in MCGM wards. Data is also available on 
>>>>>> MCGM 
>>>>>> website but it's not public. I am not a tech person and don't know how 
>>>>>> to 
>>>>>> scrape this data from website.
>>>>>>
>>>>>> Need help from anyone who knows how to deal with this.  
>>>>>>
>>>>>> Thank you,
>>>>>> Vijay
>>>>>>
>>>>>> -- 
>>>>>> Datameet is a community of Data Science enthusiasts in India. Know 
>>>>>> more about us by visiting http://datameet.org
>>>>>> --- 
>>>>>> You received this message because you are subscribed to the Google 
>>>>>> Groups "datameet" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>> send an email to datameet+u...@googlegroups.com.
>>>>>> To view this discussion on the web visit 
>>>>>> https://groups.google.com/d/msgid/datameet/CA%2BzgrCQ4Ed-vU8X4QpBGu5YD26ytH%2B3%3DQgUEbioRLNHBbM0K6g%40mail.gmail.com
>>>>>>  
>>>>>> <https://groups.google.com/d/msgid/datameet/CA%2BzgrCQ4Ed-vU8X4QpBGu5YD26ytH%2B3%3DQgUEbioRLNHBbM0K6g%40mail.gmail.com?utm_medium=email_source=footer>
>>>>>> .
>>>>>>
>>>>> -- 
>>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>>> about us by visiting http://datameet.org
>>>> --- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "datameet" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to datameet+u...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/

[datameet] Re: Agriculture filed Boundary Detection using satellite imagery

2020-07-15 Thread Souvik Sankar Mitra
You can do it using LULC masking. Classify agriculture lands and mask it. 
Then apply different convolution filters for edge detection. 

On Wednesday, July 15, 2020 at 5:37:57 PM UTC+5:30 apurva...@gmail.com 
wrote:

> Hello every one i am working on agriculture information system 
> i want to extract the the agriculture field boundary using satellite image 
> (sentinel)
> Can any one help me on this topic ?  kindly suggest me for any software , 
> tool or script. 
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/50ba1b32-bfcf-4d4a-8101-d1cec4ec8af0n%40googlegroups.com.


[datameet] Re: please help to extract data from pmgsy (pradhan matri gramin sadak yojna)

2020-06-22 Thread Souvik Sankar Mitra
I tried to get data from that site. Actually it gives data based on pixel 
position of habitats and roads for a specific block of a district. That 
pixel position changes when window size is changed. For this reason if we 
resize the window that page reloads. One way I tried is looping thorugh all 
pixels in the present window. But as my resource is limited I can't run it 
for whole page.  

The format of URL is 
- 
http://www.pmgsy-grris.nic.in/MapController.ashx?=FeatureSelection=1,1000_500=Map1

Here it is checking for (1000,500) pixel when the window size is 1366x608. 

I have written a code to check for data in (1,1) , (2,2) ,  (600,600) 
for a specific block. I got some data also saved in two csv files roads and 
habitat. 

Anybody interested can extend and modify this code. 

On Tuesday, May 5, 2020 at 10:00:07 AM UTC+5:30, Deepak Sharda wrote:
>
> dear group members
>
> We are privileged to have gems among us who know how to extract data from 
> this website 
>
> Kindly help me with data 
>
> http://www.pmgsy-grris.nic.in/gis.aspx
>
> whole of India with habitation code.
>
> I am willing to pay for shape files format.
>
> thank you 
>
> Regards 
>
>
>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/119e48f8-e115-4f64-b747-2340c4fe2359o%40googlegroups.com.
,HabitationName,District,Block,TotalPopulation,SCSTPopulation,CensusYear,Connected,ConnectedBeforePMGSY,ConnectedAfterPMGSY,PMGSYScheme,PrimarySchool,MiddleSchool,HighSchool,IntermediateSchool,ITI,DegreeCollege,HealthService,Dispaensary,MCWCenters,PHCS,VetraryHospital,TelegraphOffice,TeleConnection,BusService,RailwayStation,Electricity,BlockHQ,SubTahsil,TouristPlace,PetrolPump,Mandi,Warehouse,RetailShop,Desert,Tribal,,,
0,Pashapara,South 24-Parganas,Sonarpur,611,491,2001,Y,Y,Y,PMGSY I,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,,,
0,Chalkjagaddal,South 24-Parganas,Sonarpur,1140,953,2001,Y,Y,Y,PMGSY I,Y,N,N,N,N,N,Y,N,Y,N,N,N,Y,N,N,Y,N,N,N,N,N,N,N,N,N,,,
0,Nayapattan,South 24-Parganas,Sonarpur,665,431,2001,Y,Y,Y,PMGSY I,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,,,
0,Srirampur,South 24-Parganas,Baruipur,951,0,2001,N,N,Y,PMGSY I,N,N,N,N,N,N,N,N,Y,N,N,N,N,N,N,Y,N,N,N,N,N,N,N,N,N,,,
0,Bhurkul,South 24-Parganas,Baruipur,482,480,2001,N,N,Y,PMGSY I,Y,N,N,N,N,N,N,N,Y,N,N,N,Y,N,N,Y,N,N,N,N,N,N,N,N,N,,,
0,Madhabpur,South 24-Parganas,Baruipur,1207,1083,2001,N,N,Y,PMGSY I,Y,N,N,N,N,N,N,N,Y,N,N,N,Y,N,N,Y,N,N,N,N,N,N,N,N,N,,,
0,Fuldubi,South 24-Parganas,Baruipur,607,364,2001,Y,Y,Y,PMGSY I,Y,N,N,N,N,N,Y,N,Y,N,N,N,N,N,N,Y,N,N,N,N,N,N,N,N,N
0,Sasari,South 24-Parganas,Baruipur,1209,760,2001,N,N,Y,PMGSY I,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,,,
0,PMGSY I,T01,South 24-Parganas,Sonarpur,Sonarpur to Uttar ray pur road,0.500,10.000,9.500,Through Route,Dakshin Kalikapur(C)(Pop:1784); Hasanpur(C)(Pop:1249); Jardaha(C)(Pop:1235); Mathurapur(C)(Pop:1392); Mathurapur West(C)(Pop:1665); Murapukur(C)(Pop:1520); Ramchandrapur(C)(Pop:702)
0,PMGSY I,T01,South 24-Parganas,Sonarpur,Sonarpur to Uttar ray pur road,0.500,10.000,9.500,Through Route,Dakshin Kalikapur(C)(Pop:1784); Hasanpur(C)(Pop:1249); Jardaha(C)(Pop:1235); Mathurapur(C)(Pop:1392); Mathurapur West(C)(Pop:1665); Murapukur(C)(Pop:1520); Ramchandrapur(C)(Pop:702)
0,PMGSY I,T02,South 24-Parganas,Sonarpur,Rampur to Manikpur via Chakharinavi,0.000,6.000,6.000,Through Route,Bhabanipur(C)(Pop:3182); Bidyadharpur(C)(Pop:708); Malipukur(C)(Pop:1167); Mollapara(C)(Pop:2182); Nayapattan(C)(Pop:665); Rampur and Mukundapur(C)(Pop:869)
0,PMGSY I,T02,South 24-Parganas,Sonarpur,Rampur to Manikpur via Chakharinavi,0.000,6.000,6.000,Through Route,Bhabanipur(C)(Pop:3182); Bidyadharpur(C)(Pop:708); Malipukur(C)(Pop:1167); Mollapara(C)(Pop:2182); Nayapattan(C)(Pop:665); Rampur and Mukundapur(C)(Pop:869)
0,PMGSY I,L065,South 24-Parganas,Baruipur,South Garia Champahati Road,0.000,5.750,5.750,Link Route,
0,PMGSY I,L061,South 24-Parganas,Baruipur,Part of L061 start from Mallickpur to Ganashpur via Panchghara via Serampore via Akna Mirzapur,0.000,3.000,3.000,Link Route,Faridpur(C)(Pop:2906); Mollickpur(C)(Pop:3000); Panchghara(C)(Pop:2990); Panchghara Sarder Para(C)(Pop:2162); Pirtala(C)(Pop:3000); Purba Mallickpur(C)(Pop:2916); Srirampur(U)(Pop:951)
0,PMGSY I,L062,South 24-Parganas,Baruipur,Mallickpur PWD Road,0.750,3.750,3.000,Link Route,
0,PMGSY I,L062,South 24-Parganas,Baruipur,Part of 

Re: [datameet] Bangalore zone wise data on COVID19 in machine readable format

2020-06-19 Thread Souvik Sankar Mitra
I think its one of their security flaw. They should not publish this type 
of information in a portal. I deleted the csv from this post because it 
contains sensitive information. 

On Friday, June 19, 2020 at 11:20:51 AM UTC+5:30, Devdatta Tengshe wrote:
>
> Whoa!
>
> I can't believe that they are showing personally identifiable data on the 
> web portal..
>
> On Friday, 19 June, 2020 at 3:02:29 am UTC+5:30 souviksa...@gmail.com 
> wrote:
>
>> I scraped  https://kgis.ksrsac.in/covid/ and got list of all patients in 
>> Karnataka. 
>>
>> On Thu, Jun 18, 2020 at 5:35 PM Dilawar Singh  
>> wrote:
>>
>>> Hi DataMeet,
>>>
>>> On page 8 of this document 
>>> http://covid.bbmpgov.in/reports/report20200617.pdf there is a map which 
>>> shows location of active cases in Bangalore. Is this data available in 
>>> machine readable format?
>>>
>>> -- 
>>> Dilawar
>>> NCBS Bangalore
>>>
>>> -- 
>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>> about us by visiting http://datameet.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to datameet+u...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/datameet/CAM72-Zv8paf6PY0Lge6pT-HtUsMnAFwtMwSkrs19QVh7bedVYA%40mail.gmail.com
>>>  
>>> <https://groups.google.com/d/msgid/datameet/CAM72-Zv8paf6PY0Lge6pT-HtUsMnAFwtMwSkrs19QVh7bedVYA%40mail.gmail.com?utm_medium=email_source=footer>
>>> .
>>>
>>
>>
>> -- 
>> With Regards , 
>> Souvik Sankar Mitra,
>> *M.Tech Remote Sensing & GIS (Specialisation in Geoinformatics),*
>> *Indian Institute of Remote Sensing , Indian Space Research Organisation 
>> (ISRO),*
>> *Dehradun - 248001*
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/fae1de27-58e6-457f-bead-52c38babbb7do%40googlegroups.com.


Re: [datameet] CSR India Data

2020-06-18 Thread Souvik Sankar Mitra
You can run this script and get names of CSR with district names.

On Thu, Jun 18, 2020 at 10:29 PM Rahul Gupta 
wrote:

> Hi Datameet Community,
>
> I have to get data from the csr.gov.in website for district wise top 10
> companies and their contribution to CSR. The attached link is the district
> level map, I am struggling to open these HighCharts.js maps and fetch
> information from individual page.
>
>
> https://csr.gov.in/state.php?csr_spent_range==Arunachal%20Pradesh
>
>
> It will be a great help if someone could assist with this.
>
> Thanks & Regards-
> Rahul Gupta
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAKxLuZdymayqx43xiNrZ0MGfk%2BF%2B5Lu1RWwip2uztFV5kALOTg%40mail.gmail.com
> <https://groups.google.com/d/msgid/datameet/CAKxLuZdymayqx43xiNrZ0MGfk%2BF%2B5Lu1RWwip2uztFV5kALOTg%40mail.gmail.com?utm_medium=email_source=footer>
> .
>


-- 
With Regards ,
Souvik Sankar Mitra,
*M.Tech Remote Sensing & GIS (Specialisation in Geoinformatics),*
*Indian Institute of Remote Sensing , Indian Space Research Organisation
(ISRO),*
*Dehradun - 248001*

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM88csW4_d1EoahZMijtWUngayZkWryGYsXFBFiRetb_hrhC6w%40mail.gmail.com.


arunachal_dist.py
Description: Binary data


[datameet] Re: Help in Scraping Data

2020-05-28 Thread Souvik Sankar Mitra
I am also glad sir it helps. 

On Thursday, May 28, 2020 at 9:33:04 PM UTC+5:30, Abhiroop Sarkar wrote:
>
> Hi Souvik, this is an extremely wonderful script. I just tried this and it 
> seemed to work great in all the different scenarios. Thanks a lot for the 
> help!
>
> On Thursday, May 28, 2020 at 6:18:49 PM UTC+5:30, Souvik Sankar Mitra 
> wrote:
>>
>> To download mvt of "Mumbai Existing Land Use" I have written this code. 
>> As the mvt files are stored according to zoom level . I run it from zoom 
>> level 10 to zoom level 21. This stores all mvt files w.r.t zoom level in 
>> different directory. Hope it helps. 
>>
>> On Wednesday, May 27, 2020 at 6:32:38 PM UTC+5:30, Abhiroop Sarkar wrote:
>>>
>>> Hi, can someone help in extracting some data from 
>>> http://www.loginmumbai.org/map.html ? There is a lot of data on here 
>>> regarding Mumbai which can be helpful for further analysis.
>>>
>>> I have spent some time on this and from what I saw, all the layers are 
>>> displayed in 2 ways -
>>> 1. Directly parsing TopoJSON through OpenLayers.
>>> 2. Loading Vector Tiles (.mvt) through OpenLayers.
>>> The smaller point and polygon layers are loaded directly whereas the 
>>> heavier polygon and polyline layers are loaded through the vector tiles.
>>>
>>> What I am having trouble with is extracting the vector tiles. For 
>>> example, I want the layer under "LandUse and Development Plans" > 
>>> "Mumbai Existing Land Use". Fetching the zoomed out .mvt file and 
>>> loading it in QGIS works but zooming in shows that the edges of the 
>>> polygons are very jagged. Zooming in and then fetching the .mvt file 
>>> produces perfectly clean geometry but only for a small region (only the 
>>> specific tile).
>>>
>>> Can someone help in getting such layers in a clean way? One of the ways 
>>> would be to automate the process of panning the zoomed in webmap to 
>>> gradually load the detailed .mvt files and save them automatically upon 
>>> loading, but I lack the coding expertise to implement this. Any and all 
>>> suggestions are welcome.
>>>
>>> Thank you.
>>>
>>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/cd56a7d1-0de6-4dbb-a775-4dbe538025ce%40googlegroups.com.


[datameet] Re: Help in Scraping Data

2020-05-28 Thread Souvik Sankar Mitra
To download mvt of "Mumbai Existing Land Use" I have written this code. As 
the mvt files are stored according to zoom level . I run it from zoom level 
10 to zoom level 21. This stores all mvt files w.r.t zoom level in 
different directory. Hope it helps. 

On Wednesday, May 27, 2020 at 6:32:38 PM UTC+5:30, Abhiroop Sarkar wrote:
>
> Hi, can someone help in extracting some data from 
> http://www.loginmumbai.org/map.html ? There is a lot of data on here 
> regarding Mumbai which can be helpful for further analysis.
>
> I have spent some time on this and from what I saw, all the layers are 
> displayed in 2 ways -
> 1. Directly parsing TopoJSON through OpenLayers.
> 2. Loading Vector Tiles (.mvt) through OpenLayers.
> The smaller point and polygon layers are loaded directly whereas the 
> heavier polygon and polyline layers are loaded through the vector tiles.
>
> What I am having trouble with is extracting the vector tiles. For example, 
> I want the layer under "LandUse and Development Plans" > "Mumbai Existing 
> Land Use". Fetching the zoomed out .mvt file and loading it in QGIS works 
> but zooming in shows that the edges of the polygons are very jagged. 
> Zooming in and then fetching the .mvt file produces perfectly clean 
> geometry but only for a small region (only the specific tile).
>
> Can someone help in getting such layers in a clean way? One of the ways 
> would be to automate the process of panning the zoomed in webmap to 
> gradually load the detailed .mvt files and save them automatically upon 
> loading, but I lack the coding expertise to implement this. Any and all 
> suggestions are welcome.
>
> Thank you.
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/1e85c040-21dd-49f6-a896-97421f2d77c5%40googlegroups.com.
import math
import numpy

import requests
import os

cur = os.getcwd()


def deg2num(lat_deg, lon_deg, zoom):
  lat_rad = math.radians(lat_deg)
  n = 2.0 ** zoom
  xtile = int((lon_deg + 180.0) / 360.0 * n)
  ytile = int((1.0 - math.asinh(math.tan(lat_rad)) / math.pi) / 2.0 * n)
  return (xtile, ytile)

def unique(list1): 
arr1=[] 
# insert the list to the set 
list_set = set(list1) 
# convert the set to the list 
unique_list = (list(list_set)) 
for x in unique_list: 
arr1.append(x)
return arr1



#spatial extent of Mumbai
start = [19.308466, 72.785462]

end = [18.900295, 72.963473]

for b in range (10,22):
  print ("Processing for zoom "+str(b))
  arr=[]
  for i in numpy.arange(end[0],start[0],0.01):
  for j in numpy.arange(start[1],end[1],0.01):
  arr.append(deg2num(i,j, b))

  #print (arr)
  arr2=[]

  uni = unique(arr)


  dir1 = 'zoom_'+str(b)
  os.mkdir(dir1)
  os.chdir(cur+'/'+dir1)
  
  for a in uni:
  url = 'http://119.81.20.14:8893/mvt/mcgm_landuse/'+str(b)+'/'+str(a[0])+'/'+str(a[1])+'.mvt'
  r = requests.get(url, allow_redirects=True)
  open(str(a[0])+'_'+str(a[1])+'.mvt', 'wb').write(r.content)

  os.chdir(cur)
  
print ("Success")




Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-22 Thread Souvik Sankar Mitra
Thank you mam. I found some rates from the json. I am attaching the csv. 
But other rates are showing zero or missing values. I think Arun Ganesh sir 
can help to resolve it.  

On Saturday, May 23, 2020 at 12:20:39 AM UTC+5:30, blessing isaiah wrote:
>
> Dear,
>
> Arun Ganesh and Souvik! Thank you very much for your help. 
>
> Could we get the data of rates as well? Would be really helpful. 
>
> Best regards,
> Blessing Isaiah
>
> On Wed, 20 May, 2020, 23:40 Arun Ganesh, > 
> wrote:
>
>> Here's the raw json data which has a mess of encoded html. Maybe someone 
>> can help parse out the required details: 
>> https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e
>>
>> On Wed, May 20, 2020 at 2:03 PM Deepak Sharda > > wrote:
>>
>>> try doing web scrapping that supports locations 
>>>
>>> On Wed, May 20, 2020 at 8:04 PM blessing isaiah >> > wrote:
>>>
 Hi all,

 I would need the toll locations and the type of toll (BOT or public 
 funded etc) from here, 
 http://tis.nhai.gov.in/tollplazasonmap?language=en

 Can someone share it with me please. 
 Many thanks.

 Best regards,
 Blessing Isaiah 

 -- 
 Datameet is a community of Data Science enthusiasts in India. Know more 
 about us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google 
 Groups "datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to data...@googlegroups.com .
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/datameet/CABxwALYGs0ONwaQdEF6Cm_3eHUHVGe23fBYct2AOZ44Di0GdmQ%40mail.gmail.com
  
 
 .

>>> -- 
>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>> about us by visiting http://datameet.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to data...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/datameet/CA%2B4SsQB861Gcj3gW%2BF6hkw-H0E0%3DV9yby5Req7YgHJEpqRC8fQ%40mail.gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CA%2BGKQr3xyc_fs%3DW53rame8Q0KTYDJkaAgeR9j8P%2BWjVMTxEOHA%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/ecfe71ed-146a-4f58-aece-83aaf4760f7f%40googlegroups.com.
Tollname,Latitude,Longitude,Projecttype,Car Rate Single,Lcvrate Single,Busrate Multi,Multiaxlerate Single,Hcm Eme Single,Fourtosixexel Single,Sevenormoreexel Single
Thakurtola (End of Durg Bypass),21.117914,81.122298,BOT (Toll),95,160,325,520,520,520,520
Bankapur,14.90999,75.2798,Public Funded,55,85,180,200,285,285,350
Hirebagewadi,15.76311,74.64785,BOT (Toll),90,150,310,485,485,485,590
Durg Bypass (Dhamdanaka),21.214518,81.287962,BOT (Toll),60,120,185,275,365,275,275
Hattargi,16.1433,74.51696,Public Funded,25,45,95,100,145,145,175
Kognoli,16.5422,74.31825,Public Funded,70,115,235,260,370,370,455
Tundla,27.218247,78.272821,BOT (Toll),55,90,185,205,295,295,355
Brahamarakotlu,12.875095,75.003849,Public Funded,25,40,90,95,140,140,170
Cable Stayed Naini Bridge,25.41629,81.861017,Public Funded,20,35,75,80,115,115,140
Bassi,25.033921,74.817676,OMT,75,125,260,285,410,410,500
Aroli,25.09118,75.199063,OMT,65,105,220,240,350,350,425
Dhaneshwar,25.072621,75.598467,OMT,65,105,215,235,340,340,415
Fatehpur,25.122,76.572806,OMT,95,150,305,470,470,470,590
Simliya,25.170089,76.130552,OMT,50,85,175,270,270,270,330
Omallur,11.72034,78.07338,BOT (Toll),75,135,270,435,435,435,435
Nathakkarai,11.592809,78.780812,BOT (Toll),50,90,180,285,285,285,285
Veeracholapuram 

Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-21 Thread Souvik Sankar Mitra
Thank you very much sir. I have done it using a google sheet script. I am 
attaching the code. You have to paste the code in google script and named 
it as the function name "ParseJson".gs . To use it for any json url , type 
=ParseJson(url). It will fetch the json and arrange it it rows and colomns. 

On Thursday, May 21, 2020 at 8:40:37 AM UTC+5:30, Deepak Sharda wrote:
>
> dear souvik , 
>
> so you share what kind of parsing tool or bot you use. we all have many 
> queries like this for parsing locations from websites. it would be easier 
> for everyone if you could share guidance on this matter.
>
> thank you 
>
> On Thu, May 21, 2020 at 8:07 AM Arun Ganesh  > wrote:
>
>> Wonderful Souvik!
>>
>> Have added the CSV to the gist for future reference 
>> https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e . 
>> Would be great if you could share the code to do the parsing, it would be 
>> interesting to parse out the rates as well.
>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CA%2BGKQr2zy%3DJ1qSt-MXc6B6wvn0CAYewoe5PZWiHfyB7qD%2BQZQA%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/fbaa3dba-336e-47b3-9175-c03a63aff6fa%40googlegroups.com.
/**
 * Retrieves all the rows in the active spreadsheet that contain data and logs 
the
 * values for each row.
 */
function readRows() {
  var sheet = SpreadsheetApp.getActiveSheet();
  var rows = sheet.getDataRange();
  var numRows = rows.getNumRows();
  var values = rows.getValues();

  for (var i = 0; i <= numRows - 1; i++) {
var row = values[i];
Logger.log(row);
  }
};

/**
 * Adds a custom menu to the active spreadsheet, containing a single menu item
 * for invoking the readRows() function specified above.
 * The onOpen() function, when defined, is automatically invoked whenever the
 * spreadsheet is opened.
 */
function onOpen() {
  var sheet = SpreadsheetApp.getActiveSpreadsheet();
  var entries = [{
name : "Read Data",
functionName : "readRows"
  }];
  sheet.addMenu("Script Center Menu", entries);
};


function ParseJson(url, query, options) {
  return ImportJSONAdvanced(url, query, options, includeXPath_, 
defaultTransform_);
}




function ImportJSONAdvanced(url, query, options, includeFunc, transformFunc) {
  var jsondata = UrlFetchApp.fetch(url);
  var object   = JSON.parse(jsondata.getContentText());
  
  return parseJSONObject_(object, query, options, includeFunc, transformFunc);
}

/** 
 * Encodes the given value to use within a URL.
 *
 * @param {value} the value to be encoded
 * 
 * @return the value encoded using URL percent-encoding
 */
function URLEncode(value) {
  return encodeURIComponent(value.toString());  
}

/** 
 * Parses a JSON object and returns a two-dimensional array containing the data 
of that object.
 */
function parseJSONObject_(object, query, options, includeFunc, transformFunc) {
  var headers = new Array();
  var data= new Array();
  
  if (query && !Array.isArray(query) && query.toString().indexOf(",") != -1) {
query = query.toString().split(",");
  }
  
  if (options) {
options = options.toString().split(",");
  }

  parseData_(headers, data, "", 1, object, query, options, includeFunc);
  parseHeaders_(headers, data);
  transformData_(data, options, transformFunc);
  
  return hasOption_(options, "noHeaders") ? (data.length > 1 ? data.slice(1) : 
new Array()) : data;
}

/** 
 * Parses the data contained within the given value and inserts it into the 
data two-dimensional array starting at the rowIndex. 
 * If the data is to be inserted into a new column, a new header is added to 
the headers array. The value can be an object, 
 * array or scalar value.
 *
 * If the value is an object, it's properties are iterated through and passed 
back into this function with the name of each 
 * property extending the path. For instance, if the object contains the 
property "entry" and the path passed in was "/feed",
 * this function is called with the value of the entry property and the path 
"/feed/entry".
 *
 * If the value is an array containing other 

Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Souvik Sankar Mitra
I parse the json data and save the required details in csv. 

On Wednesday, May 20, 2020 at 11:40:46 PM UTC+5:30, Arun Ganesh wrote:
>
> Here's the raw json data which has a mess of encoded html. Maybe someone 
> can help parse out the required details: 
> https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e
>
> On Wed, May 20, 2020 at 2:03 PM Deepak Sharda  > wrote:
>
>> try doing web scrapping that supports locations 
>>
>> On Wed, May 20, 2020 at 8:04 PM blessing isaiah > > wrote:
>>
>>> Hi all,
>>>
>>> I would need the toll locations and the type of toll (BOT or public 
>>> funded etc) from here, 
>>> http://tis.nhai.gov.in/tollplazasonmap?language=en
>>>
>>> Can someone share it with me please. 
>>> Many thanks.
>>>
>>> Best regards,
>>> Blessing Isaiah 
>>>
>>> -- 
>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>> about us by visiting http://datameet.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to data...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/datameet/CABxwALYGs0ONwaQdEF6Cm_3eHUHVGe23fBYct2AOZ44Di0GdmQ%40mail.gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CA%2B4SsQB861Gcj3gW%2BF6hkw-H0E0%3DV9yby5Req7YgHJEpqRC8fQ%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/d9811457-c57b-4e5a-8fa8-fbcc1b8f20d5%40googlegroups.com.
Tollname,Latitude,Longitude,Projecttype
Thakurtola (End of Durg Bypass),21.117914,81.122298,BOT (Toll)
Bankapur,14.90999,75.2798,Public Funded
Hirebagewadi,15.76311,74.64785,BOT (Toll)
Durg Bypass (Dhamdanaka),21.214518,81.287962,BOT (Toll)
Hattargi,16.1433,74.51696,Public Funded
Kognoli,16.5422,74.31825,Public Funded
Tundla,27.218247,78.272821,BOT (Toll)
Brahamarakotlu,12.875095,75.003849,Public Funded
Cable Stayed Naini Bridge,25.41629,81.861017,Public Funded
Bassi,25.033921,74.817676,OMT
Aroli,25.09118,75.199063,OMT
Dhaneshwar,25.072621,75.598467,OMT
Fatehpur,25.122,76.572806,OMT
Simliya,25.170089,76.130552,OMT
Omallur,11.72034,78.07338,BOT (Toll)
Nathakkarai,11.592809,78.780812,BOT (Toll)
Veeracholapuram (West),11.736464,79.010409,BOT (Toll)
Palayam (Dharmapuri),12.00643,78.08042,BOT (Toll)
Rasampalayan,11.193895,78.103219,BOT (Toll)
Nemili (Sriperumbudur),12.982155,79.968944,Public Funded
Joya,28.836176,78.420749,Public Funded
"Khalghat -MP/Maharashtra Border(Sendhwa, Jamli)",21.75667,75.156824,BOT (Toll)
Boothakudi,10.64127,78.581277,OMT
Chittampatti,9.989594,78.244086,OMT
Banglore - Neelamangla,13.090749,77.44,BOT (Toll)
Chalageri,14.556778,75.728025,Public Funded
Choundha,26.467309,78.010916,OMT
Brijghat,28.767149,78.110043,Public Funded
Usaka ( Chamari),26.036608,79.577976,Public Funded
Indore Dewas (Indore Bypass),22.822692,75.938208,BOT (Toll)
Dasna,28.684924,77.522471,Public Funded
Siwaya,29.095601,77.710995,BOT (Toll)
Daffi,25.248275,82.994017,BOT (Toll)
Mohania,25.19093,83.559544,BOT (Toll)
Sasaram,24.957125,83.954997,BOT (Toll)
Anantram,26.594931,79.268034,BOT (Toll)
Tendua (Gorakhpur Bypass),26.724015,83.274398,OMT
Barajore (Bara),26.386389,80.021485,BOT (Toll)
Bharudi,21.805381,70.663976,BOT (Toll)
Pithadiya,22.082642,70.781004,BOT (Toll)
Choryasi,21.303367,72.95418,BOT (Toll)
Anewadi,17.810312,73.971381,BOT (Toll)
Mettupatti,11.66167,78.330768,BOT (Toll)
Doddakarenahalli (Neelmangala ),13.068175,77.3605999,BOT (Toll)
Thirumandurai,11.396808,78.99356,BOT (Toll)
Paduna,24.271438,73.67379,Public Funded
Patas,18.424491,74.466193,BOT (Toll)
Thandikhui,32.607394,74.944346,Public Funded
Daroda,20.450467,78.755153,OMT
Samakhiyali,23.308555,70.49059,BOT (Toll)
Kelapur,20.019307,78.54022,Public Funded
Nandgaon Peth,20.951008,77.788359,BOT (Toll)
Mundiyar,25.208647,77.136059,OMT

Re: [datameet] Get Smart Route to Avoid Containment Zone

2020-05-17 Thread Souvik Sankar Mitra
Thank you very much for your advice sir. I will try to implement of getting 
location by clicking. Actually sir I use Google's Direction API here. But I 
can try to implement it using open source routing. 
Thank you for the Stack Overflow link sir. It will help me.. 

With regards , 
Souvik Sankar Mitra
M.Tech(Geoinformatics) , Indian Institute of Remote Sensing

On Sunday, May 17, 2020 at 11:26:42 AM UTC+5:30, Nikhil VJ wrote:
>
> Hi Souvik,
>
> Good work!
>
> Suggestion on UI end: can you find a way to capture clicks or right-clicks 
> on the map and let the user add to/from addresses that way, other than 
> typing?
> That might also save on place api usage.
>
> I've made custom context menus / click-to-pin feature in Leaflet maps, not 
> sure how its done in google maps but I've seen such implementations 
> elsewhere.
> Here's a link from a quick search: 
> https://stackoverflow.com/questions/7168394/google-map-v3-context-menu#10957262
>
> Another for-later suggestion: Is your routing running on google's api or 
> other? If other, then it might be a good bet to change the ui to leaflet 
> and avoid google's usage costs that hit when visitor count increases.
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Wed, May 13, 2020 at 12:29 PM Souvik Sankar Mitra <
> souviksa...@gmail.com > wrote:
>
>> I have developed a smart navigation application to avoid containment 
>> zone. It gives a safe route between source and destination which does not 
>> go through any containment zone. Till now it only available for Kolkata and 
>> Howrah but it can be implemented for other cities also. 
>>
>> You can check it here - http://shampa.covidrespons.in/
>> For more information you can check out this video - 
>> https://www.youtube.com/watch?v=UtUaCDmJV_M
>>
>> Hope you like it. 
>> Thank you. 
>>
>> Souvik Sankar Mitra
>> M.Tech (Geoinformatics) , Indian Institute of Remote Sensing
>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/41afc5f7-aafe-4451-9d94-1d09ab15feb3%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/datameet/41afc5f7-aafe-4451-9d94-1d09ab15feb3%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/fb9b81da-98d0-4b05-9561-0084f6ce7a4f%40googlegroups.com.


[datameet] Get Smart Route to Avoid Containment Zone

2020-05-13 Thread Souvik Sankar Mitra
I have developed a smart navigation application to avoid containment zone. 
It gives a safe route between source and destination which does not go 
through any containment zone. Till now it only available for Kolkata and 
Howrah but it can be implemented for other cities also. 

You can check it here - http://shampa.covidrespons.in/
For more information you can check out this video - 
https://www.youtube.com/watch?v=UtUaCDmJV_M

Hope you like it. 
Thank you. 

Souvik Sankar Mitra
M.Tech (Geoinformatics) , Indian Institute of Remote Sensing

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/41afc5f7-aafe-4451-9d94-1d09ab15feb3%40googlegroups.com.


Re: [datameet] #COVID19 mohfw data

2020-05-01 Thread Souvik Sankar Mitra
Thank you sir.. 

On Saturday, May 2, 2020 at 2:12:21 AM UTC+5:30, Thejesh GN wrote:
>
> Thanks for letting me know Souvik. There was a bug. I have fixed it. It 
> should be updated now.
> Thej
> --
> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>
>
> On Sat, 2 May 2020 at 00:26, Souvik Sankar Mitra  > wrote:
>
>> Sir, data in 
>> https://data.thejeshgn.com/covid19/_design/india/_view/incidents?nounce=20200501
>>  is 
>> not updating for a day. 
>>
>> On Sunday, April 12, 2020 at 8:51:29 PM UTC+5:30, Thejesh GN wrote:
>>>
>>> So there is no time of publishing in the JSON.
>>>
>>> So I will pull the time from HTML and data from the JSON.
>>>
>>> Otherwise I have to keep checking for the data change!
>>>
>>> I might do hash of the JSON data as well. Every hour just to check.
>>>
>>>
>>>
>>> Regards,
>>> Thej
>>>
>>> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
>>> http://thejeshgn.com
>>>
>>> On Sun, 12 Apr, 2020, 6:58 PM Vikram Shetty,  wrote:
>>>
>>>> Thejesh and all, 
>>>>
>>>> MoHFW has new dashboard (https://www.mohfw.gov.in/dashboard/index.php) 
>>>> on site.. and now state values are now in JSON. 
>>>> https://www.mohfw.gov.in/dashboard/data/data.json
>>>>
>>>> Hope it helps.
>>>>
>>>> Regards,
>>>> Vikram
>>>>
>>>> On Apr 5, 2020, at 7:47 PM, Thejesh GN  wrote:
>>>>
>>>> I recorded a video yesterday, in which I go through each video 
>>>> explaining how they are sourced, archived and parsed. I also show some 
>>>> example usage
>>>>
>>>> https://thejeshgn.com/2020/04/05/datameets-covid-19-india-datasets/
>>>>
>>>> Also some of the members had questions about converting from JSON to 
>>>> CSV.
>>>>
>>>> You can use a simple online conversion tool like this
>>>>
>>>> https://apps.thejeshgn.com/json-csv/
>>>>
>>>> If required. I can convert the JSON to CSV on a daily basis and commit 
>>>> to the github. Let me know.
>>>>
>>>>
>>>> Thej
>>>> --
>>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>> http://thejeshgn.com
>>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>>
>>>>
>>>> On Wed, 1 Apr 2020 at 17:40, Pooja Dantewadia  
>>>> wrote:
>>>>
>>>>> Works on incognito mode or if the ad blocker is removed. 
>>>>>
>>>>> On Wed, 1 Apr 2020 at 5:37 PM, Ajeya Raj  wrote:
>>>>>
>>>>>> state wise data is not opening right now
>>>>>>
>>>>>> On Wed, Apr 1, 2020 at 4:07 PM Thejesh GN  wrote:
>>>>>>
>>>>>>> - They keep changing the format of the page everyday. I have to fix 
>>>>>>> it almost every time. Anyway latest (9am)data is up.
>>>>>>> - No updates from ICMR yet!
>>>>>>> - I also have a page where you can see COVID19 related but non virus 
>>>>>>> deaths. Sadly even that is growing. 
>>>>>>> https://thejeshgn.com/projects/covid19-india/non-virus-deaths/
>>>>>>>
>>>>>>>
>>>>>>> Thej
>>>>>>> --
>>>>>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>>>>> http://thejeshgn.com
>>>>>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 30 Mar 2020 at 19:01, Thejesh GN  wrote:
>>>>>>>
>>>>>>>> No data directly from ICMR anymore. It will come from mohfw.
>>>>>>>>
>>>>>>>> https://twitter.com/arvindgunasekar/status/1244515099001393152
>>>>>>>>
>>>>>>>> Thej
>>>>>>>> --
>>>>>>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>>>>>> http://thejeshgn.com
>>>>>>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, 30 Mar 2020 at 16:49, Ajeya Raj  wrote:
>>>>>>>>
>>>>>>>>> Do we have citywise covid-19 cases data
>>>>>>>>>
>>>>>>>>> On Sun, Mar 29, 2020 at

Re: [datameet] #COVID19 mohfw data

2020-05-01 Thread Souvik Sankar Mitra
Sir, data in 
https://data.thejeshgn.com/covid19/_design/india/_view/incidents?nounce=20200501
 is 
not updating for a day. 

On Sunday, April 12, 2020 at 8:51:29 PM UTC+5:30, Thejesh GN wrote:
>
> So there is no time of publishing in the JSON.
>
> So I will pull the time from HTML and data from the JSON.
>
> Otherwise I have to keep checking for the data change!
>
> I might do hash of the JSON data as well. Every hour just to check.
>
>
>
> Regards,
> Thej
>
> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
>
> On Sun, 12 Apr, 2020, 6:58 PM Vikram Shetty,  > wrote:
>
>> Thejesh and all, 
>>
>> MoHFW has new dashboard (https://www.mohfw.gov.in/dashboard/index.php) 
>> on site.. and now state values are now in JSON. 
>> https://www.mohfw.gov.in/dashboard/data/data.json
>>
>> Hope it helps.
>>
>> Regards,
>> Vikram
>>
>> On Apr 5, 2020, at 7:47 PM, Thejesh GN > 
>> wrote:
>>
>> I recorded a video yesterday, in which I go through each video explaining 
>> how they are sourced, archived and parsed. I also show some example usage
>>
>> https://thejeshgn.com/2020/04/05/datameets-covid-19-india-datasets/
>>
>> Also some of the members had questions about converting from JSON to CSV.
>>
>> You can use a simple online conversion tool like this
>>
>> https://apps.thejeshgn.com/json-csv/
>>
>> If required. I can convert the JSON to CSV on a daily basis and commit to 
>> the github. Let me know.
>>
>>
>> Thej
>> --
>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>> http://thejeshgn.com
>> GPG ID :  0xBFFC8DD3C06DD6B0
>>
>>
>> On Wed, 1 Apr 2020 at 17:40, Pooja Dantewadia > > wrote:
>>
>>> Works on incognito mode or if the ad blocker is removed. 
>>>
>>> On Wed, 1 Apr 2020 at 5:37 PM, Ajeya Raj >> > wrote:
>>>
 state wise data is not opening right now

 On Wed, Apr 1, 2020 at 4:07 PM Thejesh GN >>> > wrote:

> - They keep changing the format of the page everyday. I have to fix it 
> almost every time. Anyway latest (9am)data is up.
> - No updates from ICMR yet!
> - I also have a page where you can see COVID19 related but non virus 
> deaths. Sadly even that is growing. 
> https://thejeshgn.com/projects/covid19-india/non-virus-deaths/
>
>
> Thej
> --
> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>
>
> On Mon, 30 Mar 2020 at 19:01, Thejesh GN  > wrote:
>
>> No data directly from ICMR anymore. It will come from mohfw.
>>
>> https://twitter.com/arvindgunasekar/status/1244515099001393152
>>
>> Thej
>> --
>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>> http://thejeshgn.com
>> GPG ID :  0xBFFC8DD3C06DD6B0
>>
>>
>> On Mon, 30 Mar 2020 at 16:49, Ajeya Raj > > wrote:
>>
>>> Do we have citywise covid-19 cases data
>>>
>>> On Sun, Mar 29, 2020 at 11:32 PM Thejesh GN >> > wrote:
>>>
 They just removed the columns confirmed_foreign and 
 confirmed_indians. Now it's just one total number. So I have updated 
 the 
 Data and API accordingly. 

 There is one record for each state and report_time combination.

 {   "id":"2020-03-12T11:00:00.00+05:30|dl",
"key":"2020-03-12T11:00:00.00+05:30|dl",
"value":{
   "_id":"2020-03-12T11:00:00.00+05:30|dl",
   "_rev":"1-1b430a0a4b4a8b43fdab12d040ae8735",
   "state":"dl",
   "report_time":"2020-03-12T11:00:00.00+05:30",
   "confirmed_india":6,
   "confirmed_foreign":0,
   "confirmed": 6,
   "cured":0,
   "death":0,
   "source":"mohfw",
   "type":"cases"  
 }
 }

 confirmed_india and confirmed_foreign are present only when they 
 are available. Use confirmed which is usually the addition of
 confirmed_india and confirmed_foreign
 http://projects.datameet.org/covid19/mohfw/

 Thej
 --
 Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
 http://thejeshgn.com
 GPG ID :  0xBFFC8DD3C06DD6B0


 On Sun, 29 Mar 2020 at 21:51, Bodhisattwa Mandal <
 bodhisat...@gmail.com > wrote:

> There are huge discrepancies in confirmed cases at district level 
> between MoHFW, state health bulletins, media news channel, 
> covid19india 
> that it has now become a tiring task to verify and find the correct 
> one. :(
>
>
>
> On Sun, 29 Mar 2020, 20:00 Thejesh GN,  > wrote:
>
>> Last one was 27th.
>>
>> https://github.com/datameet/covid19/blob/master/icmr-backup/ICMR_website_update_27March_9AM_IST.pdf
>>
>> They might release on tomorrow (working day?).
>>
>>
>> Regards,
>> Thej
>>
>> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
>> 

Re: [datameet] Re: #COVID19 mohfw data

2020-03-27 Thread Souvik Sankar Mitra
Thank you very much sir. 

On Saturday, March 28, 2020 at 10:52:20 AM UTC+5:30, Thejesh GN wrote:
>
> Just forgot to announce on the list, Data gets cached. So if you want to 
> get the latest
>
> use the format
>
>
> https://data.thejeshgn.com/covid19/_design/india/_view/incidents?descending=false=
> 
>
> Where change the  with current date/hr. For example 2020032810
>
> then url will be
>
>
> https://data.thejeshgn.com/covid19/_design/india/_view/incidents?descending=false=2020032810
>
> And that should return you the latest.
>
> Sorry I forgot.
>
> Thej
> --
> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>
>
> On Sat, 28 Mar 2020 at 10:47, Souvik Sankar Mitra  > wrote:
>
>> Data in https://data.thejeshgn.com/covid19/_design/india/_view/incidents is 
>> not getting updated. Last update 2020-03-27T09:15:00.00+05:30
>>
>> On Saturday, March 28, 2020 at 9:43:14 AM UTC+5:30, Thejesh GN wrote:
>>>
>>> Back.
>>>
>>> Regards,
>>> Thej
>>>
>>> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
>>> http://thejeshgn.com
>>>
>>> On Sat, 28 Mar, 2020, 9:32 AM Bodhisattwa Mandal,  
>>> wrote:
>>>
>>>> The MoHFW has removed the stat section today. Anyone have any idea, 
>>>> where it might be now?
>>>>
>>>> Bodhisattwa
>>>>
>>>> On Fri, 27 Mar 2020, 19:26 Thejesh GN,  wrote:
>>>>
>>>>> From 30Th Jan to 10 March MoHFW declared numbers through press 
>>>>> releases. I have them backed up now.
>>>>>
>>>>> https://github.com/datameet/covid19/tree/master/mohfw-pib-backup
>>>>>
>>>>> I will add it to the JSON data  later today.
>>>>>
>>>>>
>>>>> Thej
>>>>> --
>>>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>>> http://thejeshgn.com
>>>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>>>
>>>>>
>>>>> On Thu, 26 Mar 2020 at 04:39, Pranit Gupta  
>>>>> wrote:
>>>>>
>>>>>> Hello, I am looking for the following information by state/district: 
>>>>>> total number of tests, positive tests, negative tests. Is that 
>>>>>> information 
>>>>>> available somewhere?
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>> On Friday, March 13, 2020 at 5:36:23 PM UTC+5:30, Thejesh GN wrote:
>>>>>>>
>>>>>>>
>>>>>>> - I have been scraping and diffing the #COVID19 data at 
>>>>>>> https://t.co/qhdbOGGaeh  every 12 hours
>>>>>>>
>>>>>>> - Converting them into clean JSON and made a simple API
>>>>>>>
>>>>>>> - You can use the below URL if you like
>>>>>>>
>>>>>>>
>>>>>>> https://data.thejeshgn.com/covid19/_design/india/_view/incidents?include_docs=true=5=true
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Thej
>>>>>>>
>>>>>>> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
>>>>>>> http://thejeshgn.com
>>>>>>>
>>>>>> -- 
>>>>>> Datameet is a community of Data Science enthusiasts in India. Know 
>>>>>> more about us by visiting http://datameet.org
>>>>>> --- 
>>>>>> You received this message because you are subscribed to the Google 
>>>>>> Groups "datameet" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>> send an email to data...@googlegroups.com.
>>>>>> To view this discussion on the web visit 
>>>>>> https://groups.google.com/d/msgid/datameet/ab715892-158d-4877-9384-23735107176c%40googlegroups.com
>>>>>>  
>>>>>> <https://groups.google.com/d/msgid/datameet/ab715892-158d-4877-9384-23735107176c%40googlegroups.com?utm_medium=email_source=footer>
>>>>>> .
>>>>>>
>>>>> -- 
>>>>> Datameet is a community of Data Science enthusiasts in India. Know 
>>>>> more about us by visiting http://datameet.org
>>>>> --- 
>>>>> You received this message because you are subscribed to the Google 
>>>>

Re: [datameet] Re: #COVID19 mohfw data

2020-03-27 Thread Souvik Sankar Mitra
Data in https://data.thejeshgn.com/covid19/_design/india/_view/incidents is 
not getting updated. Last update 2020-03-27T09:15:00.00+05:30

On Saturday, March 28, 2020 at 9:43:14 AM UTC+5:30, Thejesh GN wrote:
>
> Back.
>
> Regards,
> Thej
>
> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
>
> On Sat, 28 Mar, 2020, 9:32 AM Bodhisattwa Mandal,  > wrote:
>
>> The MoHFW has removed the stat section today. Anyone have any idea, where 
>> it might be now?
>>
>> Bodhisattwa
>>
>> On Fri, 27 Mar 2020, 19:26 Thejesh GN, > 
>> wrote:
>>
>>> From 30Th Jan to 10 March MoHFW declared numbers through press releases. 
>>> I have them backed up now.
>>>
>>> https://github.com/datameet/covid19/tree/master/mohfw-pib-backup
>>>
>>> I will add it to the JSON data  later today.
>>>
>>>
>>> Thej
>>> --
>>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>>> http://thejeshgn.com
>>> GPG ID :  0xBFFC8DD3C06DD6B0
>>>
>>>
>>> On Thu, 26 Mar 2020 at 04:39, Pranit Gupta >> > wrote:
>>>
 Hello, I am looking for the following information by state/district: 
 total number of tests, positive tests, negative tests. Is that information 
 available somewhere?

 Thank you!

 On Friday, March 13, 2020 at 5:36:23 PM UTC+5:30, Thejesh GN wrote:
>
>
> - I have been scraping and diffing the #COVID19 data at 
> https://t.co/qhdbOGGaeh  every 12 hours
>
> - Converting them into clean JSON and made a simple API
>
> - You can use the below URL if you like
>
>
> https://data.thejeshgn.com/covid19/_design/india/_view/incidents?include_docs=true=5=true
>
>
>
> Regards,
> Thej
>
> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
>
 -- 
 Datameet is a community of Data Science enthusiasts in India. Know more 
 about us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google 
 Groups "datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to data...@googlegroups.com .
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/datameet/ab715892-158d-4877-9384-23735107176c%40googlegroups.com
  
 
 .

>>> -- 
>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>> about us by visiting http://datameet.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to data...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/datameet/CAABnYsXmE6x-PtNyYcKkvy9tay%2Bp0uTCdtKfSbG669k3Dmqj%2Bg%40mail.gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CAHyrfgZjquKBRVB5F8A1KcdhCFwPJmL9NPbA0PMqiLgUJ%2BvC2g%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/772fa80c-1d50-460c-9b46-2125f78fb2d8%40googlegroups.com.


Re: [datameet] Re: #COVID19 mohfw data

2020-03-24 Thread Souvik Sankar Mitra
Please update 
https://data.thejeshgn.com/covid19/_design/india/_view/incidents . Its is 
last updated in 2020-03-23T10:30:00.00+05:30

On Monday, March 23, 2020 at 6:40:49 PM UTC+5:30, Thejesh GN wrote:
>
> We were 236 as of 20th.  433 today. 
> It's almost doubling every 3-4 days now. It was 5 days before.
>
> I also made a simple page - https://thejeshgn.com/projects/covid19-india/
>
> Its my one graph to look at.
>
>
>
>
> Thej
> --
> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>
>
> On Sun, 22 Mar 2020 at 21:04, Thejesh GN > 
> wrote:
>
>> This is really good.
>>
>> It would be great if you can write a short note on how to use the data 
>> from https://www.wikidata.org/wiki/Q84055514
>>
>>
>> Thej
>> --
>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>> http://thejeshgn.com
>> GPG ID :  0xBFFC8DD3C06DD6B0
>>
>>
>> On Sun, 22 Mar 2020 at 12:14, Bodhisattwa Mandal > > wrote:
>>
>>> Hi,
>>>
>>> We are updating COVID-19 situation in India on Wikidata.
>>>
>>>
>>> https://www.wikidata.org/wiki/Wikidata:WikiProject_India/COVID-19_task_force
>>>
>>> Regards,
>>> Bodhisattwa
>>>
>>> On Sun, 22 Mar 2020, 10:59 Guneet Narula, >> > wrote:
>>>
 For the map I am maintaining, the data comes from a google sheet 
 https://docs.google.com/spreadsheets/d/e/2PACX-1vRlSCAn1nS4h9n9Fp25iuOsH54RfMUjj3xX5CZqjGUqYCVXgwgtJojuqVeqekazs2TkSJ95Jwplo7lL/pubhtml#

 That cluster map is live here https://cluster.covid19india.org/ and is 
 w.i.p. I might be integrating the district map to their dashboard at some 
 point.

 Thanks for working on the MoHFW data archive. Will be more and more 
 useful in the coming weeks. 

 I genuinely can't believe that we went from 200k to 300k cases 
 worldwide in such a short time.

 Best,
 Guneet
 On 21/03/20 8:13 am, Thejesh GN wrote:

 This is a dashboard of network connections and clusters to track 
 outbreak and transmission COVID19 in India

 https://covid19india-network.now.sh/ 

 @guneet
 Can you share you Google sheet (that drives your app) ?


 Regards,
 Thej

 Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
 http://thejeshgn.com

 On Sat, 21 Mar, 2020, 8:05 PM Thejesh GN, >>> > wrote:

> Looks like it's private. Not able to access. 
>
> Regards,
> Thej
>
> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
>
> On Sat, 21 Mar, 2020, 3:54 PM Saketha Ramanujam, <
> saketh.r...@gmail.com > wrote:
>
>> I found this sheet, don't know how authentic it is.
>>
>> https://docs.google.com/spreadsheets/d/1nzXUdaIWC84QipdVGUKTiCSc5xntBbpMpzLm6Si33zk/htmlview?sle=true#
>>
>> --
>> Saketh
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know 
>> more about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google 
>> Groups "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CAHzVmyuk7zO_sxTSSwdd-P6c%3DsOqKt%3DwbsbCvQUcEDj%2BF%2BgY9A%40mail.gmail.com
>>  
>> 
>> .
>>
> -- 
 Datameet is a community of Data Science enthusiasts in India. Know more 
 about us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to a topic in the 
 Google Groups "datameet" group.
 To unsubscribe from this topic, visit 
 https://groups.google.com/d/topic/datameet/_HnOB5iyEx0/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to 
 data...@googlegroups.com .
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/datameet/CAABnYsU7d4DrUVXPMkbiF6am%3DHDuvLDDRQ%3Dm6AJ4eXtxKJ7otg%40mail.gmail.com
  
 
 .

 -- sputznik.com / 9717664996keybase.io/guneetnarula

 -- 
 Datameet is a community of Data Science enthusiasts in India. Know more 
 about us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google 
 Groups "datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to data...@googlegroups.com .
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/datameet/3fcb33c2-5143-8a4c-cf3f-abad8f4781d6%40sputznik.com