[datameet] DATA

2022-10-14 Thread abhaysi...@gmail.com 
hey eveyone , i need a data set of any general store, medical store  or any 
restaurant but i dont need from internet which is avilable on kaggle or 
geeks for geeks so anyone has so please provide me right now i am dong BS 
in data science form IIT Madras so i have to make a BDM project where i 
have to take a data and make a model for it so please anyone has dada 
geniune one plz provide me 
Thank you  

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/38bfd62a-0ba4-443a-8071-2a03e7d828e1n%40googlegroups.com.

Re: [datameet] Rainfall data

2022-10-14 Thread Nikhil VJ 
Hi Pankaj,

Added in the webpage now:

And even after that, for some locations there just isn't data for certain
years or dates in the original dataset. Those slots were filled with junk
values in the .grd files like -999 - they've been sanitised out from here.


You just need to keep looking for a location that gives you data and let go
of the ones that don't, that is all. Not much anybody can do if IMD itself
didn't archive it.

Please also see the other instructions in the page.
0.5 grid : select accordingly on the map.

For Assam : you'll have to select the grid points coming within assam, and
try for them.

For bulk work, Generate 1 link, then copy the same structures to create
more links with the year value in the URL changed.

--
Cheers,
Nikhil VJ
https://nikhilvj.co.in


On Fri, Oct 14, 2022 at 9:12 PM Pankaj pandey  wrote:

> Dear Nikhil I am trying to download  temperature data  for 2000 of Assam
> and getting this massage: {"success":false,"message":"No data for give
> location / year"}. What is problem Please how to download temperature data
> at 0.5 degree for all grid point of assam
>
> On Fri, Oct 14, 2022 at 5:41 PM Nikhil VJ  wrote:
>
>> Hi All,
>>
>> Got a request regd need to simplify this IMD grid data fetching process
>> for folks, and that gave me impetus to complete setting up a website where
>> individual location + year data can be downloaded as simple flat CSV (which
>> opens in Excel / LibreOffice Calc).
>>
>> So here it is: *https://server.nikhilvj.co.in/imd_data/
>> *
>>
>> There's a map and year selector to help choose, after that a link is
>> generated for downloading. Open that in browser or do wget/curl
>> on commandline, and it'll download a CSV with proper naming.
>>
>> Sample link:
>>
>> https://server.nikhilvj.co.in/imd_data/API/fetchData?year=2010=25.25=76.25=all
>>
>> You are *encouraged to scrape* : use the format above to compose your
>> own link. (Hint: Excel formulas like: =A2&"?year=" are perfect for
>> this)
>>
>> With a caveat: Given that it's not some fancy server that can take huge
>> loads, there's a cruel and harsh rate limiter in place which will allow
>> only 1 hit per 10 secs per IP address for anonymous users.
>>
>> If you want to bypass that, there's an email-otp login process given
>> which will add a token and let you do 5 hits per second, and that won't be
>> restricted by IP address.
>> This would be useful in institutions where multiple users on the same
>> network may want to fetch data at the same time.
>> ex: a school/college hands-on session in climate data visualization where
>> the host as usual forgets to pre-download the data
>> So you can scrape and I can keep my server from crashing - both sides win.
>>
>> Source code is at https://github.com/answerquest/IMD-grid-data-work (see
>> imd_data_api folder)
>>
>>
>> --
>> Cheers,
>> Nikhil VJ
>> https://nikhilvj.co.in
>>
>>
>> On Wed, Oct 12, 2022 at 8:30 AM Nikhil VJ  wrote:
>>
>>> Hi Pankaj,
>>>
>>> The place initially referred to is archival data - by definition it's
>>> historic and not realtime.
>>> What I do see on https://imdpune.gov.in/lrfindex.php is "Gridded Data
>>> Real Time" -> you might want to explore that.
>>> Over there, Daily 0.5x0.5 grid temp data links are there.
>>> You can select a single date there and download the file.
>>>
>>> I don't know right now how to open it, though. imdlib library that I've
>>> used - I didn't see anything about daily data.
>>> Here's its documentation:
>>> https://imdlib.readthedocs.io/en/latest/Usage.html
>>>
>>> I've posted a query about it in the lib's github :
>>> https://github.com/iamsaswata/imdlib/issues/17 , let's see if the
>>> author responds. Perhaps some inspection of the source code also might
>>> yield a way to open the daily data.
>>>
>>> PS: Next time do initiate a separate conversation with a different
>>> subject line - as here you have asked about temperature data but the
>>> subject reads rainfall data. Becomes less discoverable over the open
>>> internet or after a long period of time.
>>>
>>> --
>>> Cheers,
>>> Nikhil VJ
>>> https://nikhilvj.co.in
>>>
>>>
>>> On Thu, Oct 6, 2022 at 11:56 PM Pankaj pandey 
>>> wrote:
>>>
 Thanks Akshay for sharing the code. Actually I want to download tmax
 and tmin data at 0.5 degree not at 1 degree for 2000 to 2021. As I want to
 dowanload  gridded data  real time Download Daily Maximum Temperature
 data ( 0.5 x 0.5 degree) Binary File (Real-time). In this page only
 the select data option is available. Should i  replace year with data  in
 code.
 It is requested please modify code for this case

 with regards
 p.k.pANDEY



 On Thu, Oct 6, 2022 at 11:30 PM Akshay Kumar 
 wrote:

> Dear Sir
> Pl check this link.
>
> https://github.com/Manisht9/imdData
>
>
> On Thu, 6 Oct 2022 at 10:55 PM, Pankaj pandey 
> wrote:
>
>> Dear  Nikhil  how can batch

Re: [datameet] Rainfall data

2022-10-14 Thread Pankaj pandey 
Dear Nikhil I am trying to download  temperature data  for 2000 of Assam
and getting this massage: {"success":false,"message":"No data for give
location / year"}. What is problem Please how to download temperature data
at 0.5 degree for all grid point of assam

On Fri, Oct 14, 2022 at 5:41 PM Nikhil VJ  wrote:

> Hi All,
>
> Got a request regd need to simplify this IMD grid data fetching process
> for folks, and that gave me impetus to complete setting up a website where
> individual location + year data can be downloaded as simple flat CSV (which
> opens in Excel / LibreOffice Calc).
>
> So here it is: *https://server.nikhilvj.co.in/imd_data/
> *
>
> There's a map and year selector to help choose, after that a link is
> generated for downloading. Open that in browser or do wget/curl
> on commandline, and it'll download a CSV with proper naming.
>
> Sample link:
>
> https://server.nikhilvj.co.in/imd_data/API/fetchData?year=2010=25.25=76.25=all
>
> You are *encouraged to scrape* : use the format above to compose your own
> link. (Hint: Excel formulas like: =A2&"?year=" are perfect for this)
>
> With a caveat: Given that it's not some fancy server that can take huge
> loads, there's a cruel and harsh rate limiter in place which will allow
> only 1 hit per 10 secs per IP address for anonymous users.
>
> If you want to bypass that, there's an email-otp login process given which
> will add a token and let you do 5 hits per second, and that won't be
> restricted by IP address.
> This would be useful in institutions where multiple users on the same
> network may want to fetch data at the same time.
> ex: a school/college hands-on session in climate data visualization where
> the host as usual forgets to pre-download the data
> So you can scrape and I can keep my server from crashing - both sides win.
>
> Source code is at https://github.com/answerquest/IMD-grid-data-work (see
> imd_data_api folder)
>
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Wed, Oct 12, 2022 at 8:30 AM Nikhil VJ  wrote:
>
>> Hi Pankaj,
>>
>> The place initially referred to is archival data - by definition it's
>> historic and not realtime.
>> What I do see on https://imdpune.gov.in/lrfindex.php is "Gridded Data
>> Real Time" -> you might want to explore that.
>> Over there, Daily 0.5x0.5 grid temp data links are there.
>> You can select a single date there and download the file.
>>
>> I don't know right now how to open it, though. imdlib library that I've
>> used - I didn't see anything about daily data.
>> Here's its documentation:
>> https://imdlib.readthedocs.io/en/latest/Usage.html
>>
>> I've posted a query about it in the lib's github :
>> https://github.com/iamsaswata/imdlib/issues/17 , let's see if the author
>> responds. Perhaps some inspection of the source code also might yield a way
>> to open the daily data.
>>
>> PS: Next time do initiate a separate conversation with a different
>> subject line - as here you have asked about temperature data but the
>> subject reads rainfall data. Becomes less discoverable over the open
>> internet or after a long period of time.
>>
>> --
>> Cheers,
>> Nikhil VJ
>> https://nikhilvj.co.in
>>
>>
>> On Thu, Oct 6, 2022 at 11:56 PM Pankaj pandey 
>> wrote:
>>
>>> Thanks Akshay for sharing the code. Actually I want to download tmax and
>>> tmin data at 0.5 degree not at 1 degree for 2000 to 2021. As I want to
>>> dowanload  gridded data  real time Download Daily Maximum Temperature
>>> data ( 0.5 x 0.5 degree) Binary File (Real-time). In this page only the
>>> select data option is available. Should i  replace year with data  in code.
>>> It is requested please modify code for this case
>>>
>>> with regards
>>> p.k.pANDEY
>>>
>>>
>>>
>>> On Thu, Oct 6, 2022 at 11:30 PM Akshay Kumar 
>>> wrote:
>>>
 Dear Sir
 Pl check this link.

 https://github.com/Manisht9/imdData


 On Thu, 6 Oct 2022 at 10:55 PM, Pankaj pandey 
 wrote:

> Dear  Nikhil  how can batch download data,
> https://imdpune.gov.in/lrfindex.php. This page give data wise data.
> how can download  suppose 20 year data of temperature. Any python code to
> do this to bulk download  in csv. Please update any code will very much
> neede
>
> With Regards
> P.K.Pandey
>
>
> On Thu, Oct 6, 2022 at 7:28 PM Nikhil VJ  wrote:
>
>> Hi Georgelio, Vastav,
>>
>> While this link is down:
>> https://imdpune.gov.in/Clim_Pred_LRF_New/Grided_Data_Download.html
>>
>> Upon browsing around the IMD site I found under Climate Monitoring in
>> top menu, https://imdpune.gov.in/lrfindex.php
>> Side menu: Gridded Data Archive -> All the data is there and
>> available just fine.
>>
>>
>> So, taking offline my data dump to save my server's bandwidth.
>>
>>
>> --
>> Cheers,
>> Nikhil VJ
>> https://nikhilvj.co.in
>>
>>
>> On Thu, Oct 6, 2022 at 6:38

[datameet] India boundary and Inland waters

2022-10-14 Thread Deepak Sharda 
Hello GUys,

Could anyone lead to the best source where most updated India country 
polygon with Inland waters can be sourced. 

Thank you.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/275856fb-c6e5-4cc7-88f2-7d9886c34fb5n%40googlegroups.com.

Re: [datameet] Rainfall data

2022-10-14 Thread Nikhil VJ 
Hi All,

Got a request regd need to simplify this IMD grid data fetching process for
folks, and that gave me impetus to complete setting up a website where
individual location + year data can be downloaded as simple flat CSV (which
opens in Excel / LibreOffice Calc).

So here it is: *https://server.nikhilvj.co.in/imd_data/
*

There's a map and year selector to help choose, after that a link is
generated for downloading. Open that in browser or do wget/curl
on commandline, and it'll download a CSV with proper naming.

Sample link:
https://server.nikhilvj.co.in/imd_data/API/fetchData?year=2010=25.25=76.25=all

You are *encouraged to scrape* : use the format above to compose your own
link. (Hint: Excel formulas like: =A2&"?year=" are perfect for this)

With a caveat: Given that it's not some fancy server that can take huge
loads, there's a cruel and harsh rate limiter in place which will allow
only 1 hit per 10 secs per IP address for anonymous users.

If you want to bypass that, there's an email-otp login process given which
will add a token and let you do 5 hits per second, and that won't be
restricted by IP address.
This would be useful in institutions where multiple users on the same
network may want to fetch data at the same time.
ex: a school/college hands-on session in climate data visualization where
the host as usual forgets to pre-download the data
So you can scrape and I can keep my server from crashing - both sides win.

Source code is at https://github.com/answerquest/IMD-grid-data-work (see
imd_data_api folder)


--
Cheers,
Nikhil VJ
https://nikhilvj.co.in


On Wed, Oct 12, 2022 at 8:30 AM Nikhil VJ  wrote:

> Hi Pankaj,
>
> The place initially referred to is archival data - by definition it's
> historic and not realtime.
> What I do see on https://imdpune.gov.in/lrfindex.php is "Gridded Data
> Real Time" -> you might want to explore that.
> Over there, Daily 0.5x0.5 grid temp data links are there.
> You can select a single date there and download the file.
>
> I don't know right now how to open it, though. imdlib library that I've
> used - I didn't see anything about daily data.
> Here's its documentation:
> https://imdlib.readthedocs.io/en/latest/Usage.html
>
> I've posted a query about it in the lib's github :
> https://github.com/iamsaswata/imdlib/issues/17 , let's see if the author
> responds. Perhaps some inspection of the source code also might yield a way
> to open the daily data.
>
> PS: Next time do initiate a separate conversation with a different subject
> line - as here you have asked about temperature data but the subject reads
> rainfall data. Becomes less discoverable over the open internet or after a
> long period of time.
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Thu, Oct 6, 2022 at 11:56 PM Pankaj pandey  wrote:
>
>> Thanks Akshay for sharing the code. Actually I want to download tmax and
>> tmin data at 0.5 degree not at 1 degree for 2000 to 2021. As I want to
>> dowanload  gridded data  real time Download Daily Maximum Temperature
>> data ( 0.5 x 0.5 degree) Binary File (Real-time). In this page only the
>> select data option is available. Should i  replace year with data  in code.
>> It is requested please modify code for this case
>>
>> with regards
>> p.k.pANDEY
>>
>>
>>
>> On Thu, Oct 6, 2022 at 11:30 PM Akshay Kumar 
>> wrote:
>>
>>> Dear Sir
>>> Pl check this link.
>>>
>>> https://github.com/Manisht9/imdData
>>>
>>>
>>> On Thu, 6 Oct 2022 at 10:55 PM, Pankaj pandey 
>>> wrote:
>>>
 Dear  Nikhil  how can batch download data,
 https://imdpune.gov.in/lrfindex.php. This page give data wise data.
 how can download  suppose 20 year data of temperature. Any python code to
 do this to bulk download  in csv. Please update any code will very much
 neede

 With Regards
 P.K.Pandey


 On Thu, Oct 6, 2022 at 7:28 PM Nikhil VJ  wrote:

> Hi Georgelio, Vastav,
>
> While this link is down:
> https://imdpune.gov.in/Clim_Pred_LRF_New/Grided_Data_Download.html
>
> Upon browsing around the IMD site I found under Climate Monitoring in
> top menu, https://imdpune.gov.in/lrfindex.php
> Side menu: Gridded Data Archive -> All the data is there and available
> just fine.
>
>
> So, taking offline my data dump to save my server's bandwidth.
>
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Thu, Oct 6, 2022 at 6:38 PM Nikhil VJ  wrote:
>
>> Wow, Dead Internet Theory vibes 蘿
>>
>> Ok, I've moved all the gridded data file copies that I had at my end
>> to: https://server.nikhilvj.co.in/dump/imd_grid/
>>
>> Disclaimer: they were download from  source
>> 
>> on 13th July 2022, and their sha256 checksums are at:
>>
>>

Re: [datameet] Digest for datameet@googlegroups.com - 2 updates in 1 topic

2022-10-14 Thread Devdatta Tengshe 
Hi All,

Looks like this data is publicly available 
here: https://data.gov.in/resource/shapefile-rivers

Regards,
Devdatta Tengshe

On Friday, 23 September, 2022 at 10:34:54 am UTC+5:30 Devdatta Tengshe 
wrote:

> Hi Ravi,
>
> While I haven't scraped the data so far, it should be easy enough to do 
> it, since it is available at this endpoint: 
> https://arc.indiawris.gov.in/server/rest/services/SubInfoSysLCC/River_StreamOrder/MapServer
>
>
> Regards,
> Devdatta
>
>
> On Thu, 22 Sept 2022 at 20:59, Ravi Chopra  wrote:
>
>> Hi Devdatta,
>> By any chance you have river line of India may be from WRIS? Or any other 
>> reliable source?
>>
>> I am looking for rivers of India.
>>
>> Thanks
>> Ravi
>>
>> On Wed, 24 Aug 2022, 05:54 ,  wrote:
>>
>>> data...@googlegroups.com 
>>> 
>>>  Google 
>>> Groups 
>>> 
>>>  
>>> 
>>>  
>>> Topic digest 
>>> View all topics 
>>> 
>>>  
>>>
>>>- Looking for a Cauvery River basin shape file 
>>>
>>> <#m_8394149301659215080_m_6476766470296744709_m_7570434743661773979_group_thread_0>
>>>  
>>>- 2 Updates 
>>>
>>> Looking for a Cauvery River basin shape file 
>>> 
>>>  
>>> Arun Ganesh : Aug 22 10:55PM -0400 
>>>
>>> The post also reminded me that some time ago it was possible to download
>>> vectors from some of the Bhuvan geoservers as a GeoRSS XML
>>>
>>> https://github.com/planemad/qgis-extract-html-attributes/issues/1#issue-790240586
>>> .
>>>  
>>> On Mon, Aug 22, 2022 at 4:30 PM Dammalapati Sai Krishna <
>>> Devdatta Tengshe : Aug 23 09:31AM +0530 
>>>
>>> I had scraped this data from India Wris, some years back.
>>>  
>>> Here is a link to the Kaveri Basin, in GeoJSON format:
>>> https://gist.github.com/devdattaT/40fbc09ae48402157d0984df6e9f2b02
>>>  
>>>  
>>> Regards,
>>> Devdatta
>>>  
>>>  
>>> Back to top 
>>> <#m_8394149301659215080_m_6476766470296744709_m_7570434743661773979_digest_top>
>>>  
>>> You received this digest because you're subscribed to updates for this 
>>> group. You can change your settings on the group membership page 
>>> 
>>> .
>>> To unsubscribe from this group and stop receiving emails from it send an 
>>> email to datameet+u...@googlegroups.com. 
>>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to datameet+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CAPBY68KFgyZqa2vbBQ9JPqNvt3%3DsKvbUN%2BGu130DUpRn6Dg7ow%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/70c4e244-9652-4fae-99ce-7214164ca56fn%40googlegroups.com.