Re: [datameet] Rainfall data

[Akshay Kumar]

Dear Sir,
The code link that I sent has to be changed for tmax and tmin parameters.
Once it is done, you can have the csv files, which can be clipped further
with the desired area.

https://github.com/Manisht9/imdData





*Profound RegardsAkshay Kumar*

*LinkedIn *
*+91-8198834098*

P Please don't print this e-mail unless you really need to. Go Green.
*Every 3000 A4 Size papers costs 1 tree.*


On Sat, Oct 15, 2022 at 12:05 PM Pankaj pandey  wrote:

> Dear Nikhil still has the same issue for Assam.  I tried to download at  
> 26.5,93.0
> for 2001 temperature  data again getting a similar message of no data.
> PLease check the issue
>
> With regards
>
> On Fri, Oct 14, 2022 at 11:55 PM Nikhil VJ  wrote:
>
>> Hi Pankaj,
>>
>> Added in the webpage now:
>>
>> And even after that, for some locations there just isn't data for certain
>> years or dates in the original dataset. Those slots were filled with junk
>> values in the .grd files like -999 - they've been sanitised out from here.
>>
>>
>> You just need to keep looking for a location that gives you data and let
>> go of the ones that don't, that is all. Not much anybody can do if IMD
>> itself didn't archive it.
>>
>> Please also see the other instructions in the page.
>> 0.5 grid : select accordingly on the map.
>>
>> For Assam : you'll have to select the grid points coming within assam,
>> and try for them.
>>
>> For bulk work, Generate 1 link, then copy the same structures to create
>> more links with the year value in the URL changed.
>>
>> --
>> Cheers,
>> Nikhil VJ
>> https://nikhilvj.co.in
>>
>>
>> On Fri, Oct 14, 2022 at 9:12 PM Pankaj pandey 
>> wrote:
>>
>>> Dear Nikhil I am trying to download  temperature data  for 2000 of Assam
>>> and getting this massage: {"success":false,"message":"No data for give
>>> location / year"}. What is problem Please how to download temperature data
>>> at 0.5 degree for all grid point of assam
>>>
>>> On Fri, Oct 14, 2022 at 5:41 PM Nikhil VJ  wrote:
>>>
 Hi All,

 Got a request regd need to simplify this IMD grid data fetching process
 for folks, and that gave me impetus to complete setting up a website where
 individual location + year data can be downloaded as simple flat CSV (which
 opens in Excel / LibreOffice Calc).

 So here it is: *https://server.nikhilvj.co.in/imd_data/
 *

 There's a map and year selector to help choose, after that a link is
 generated for downloading. Open that in browser or do wget/curl
 on commandline, and it'll download a CSV with proper naming.

 Sample link:

 https://server.nikhilvj.co.in/imd_data/API/fetchData?year=2010=25.25=76.25=all

 You are *encouraged to scrape* : use the format above to compose your
 own link. (Hint: Excel formulas like: =A2&"?year=" are perfect for
 this)

 With a caveat: Given that it's not some fancy server that can take huge
 loads, there's a cruel and harsh rate limiter in place which will allow
 only 1 hit per 10 secs per IP address for anonymous users.

 If you want to bypass that, there's an email-otp login process given
 which will add a token and let you do 5 hits per second, and that won't be
 restricted by IP address.
 This would be useful in institutions where multiple users on the same
 network may want to fetch data at the same time.
 ex: a school/college hands-on session in climate data visualization
 where the host as usual forgets to pre-download the data
 So you can scrape and I can keep my server from crashing - both sides
 win.

 Source code is at https://github.com/answerquest/IMD-grid-data-work
 (see imd_data_api folder)


 --
 Cheers,
 Nikhil VJ
 https://nikhilvj.co.in


 On Wed, Oct 12, 2022 at 8:30 AM Nikhil VJ  wrote:

> Hi Pankaj,
>
> The place initially referred to is archival data - by definition it's
> historic and not realtime.
> What I do see on https://imdpune.gov.in/lrfindex.php is "Gridded Data
> Real Time" -> you might want to explore that.
> Over there, Daily 0.5x0.5 grid temp data links are there.
> You can select a single date there and download the file.
>
> I don't know right now how to open it, though. imdlib library that
> I've used - I didn't see anything about daily data.
> Here's its documentation:
> https://imdlib.readthedocs.io/en/latest/Usage.html
>
> I've posted a query about it in the lib's github :
> https://github.com/iamsaswata/imdlib/issues/17 , let's see if the
> author responds. Perhaps some inspection of the source code also might
> yield a way to open the daily data.
>
> PS: Next time do initiate a separate conversation with a different
> subject line - as here you have asked about temperature data but the
> subject reads 

Re: [datameet] Rainfall data

[Pankaj pandey]

Dear Nikhil still has the same issue for Assam.  I tried to download
at  26.5,93.0
for 2001 temperature  data again getting a similar message of no data.
PLease check the issue

With regards

On Fri, Oct 14, 2022 at 11:55 PM Nikhil VJ  wrote:

> Hi Pankaj,
>
> Added in the webpage now:
>
> And even after that, for some locations there just isn't data for certain
> years or dates in the original dataset. Those slots were filled with junk
> values in the .grd files like -999 - they've been sanitised out from here.
>
>
> You just need to keep looking for a location that gives you data and let
> go of the ones that don't, that is all. Not much anybody can do if IMD
> itself didn't archive it.
>
> Please also see the other instructions in the page.
> 0.5 grid : select accordingly on the map.
>
> For Assam : you'll have to select the grid points coming within assam, and
> try for them.
>
> For bulk work, Generate 1 link, then copy the same structures to create
> more links with the year value in the URL changed.
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Fri, Oct 14, 2022 at 9:12 PM Pankaj pandey  wrote:
>
>> Dear Nikhil I am trying to download  temperature data  for 2000 of Assam
>> and getting this massage: {"success":false,"message":"No data for give
>> location / year"}. What is problem Please how to download temperature data
>> at 0.5 degree for all grid point of assam
>>
>> On Fri, Oct 14, 2022 at 5:41 PM Nikhil VJ  wrote:
>>
>>> Hi All,
>>>
>>> Got a request regd need to simplify this IMD grid data fetching process
>>> for folks, and that gave me impetus to complete setting up a website where
>>> individual location + year data can be downloaded as simple flat CSV (which
>>> opens in Excel / LibreOffice Calc).
>>>
>>> So here it is: *https://server.nikhilvj.co.in/imd_data/
>>> *
>>>
>>> There's a map and year selector to help choose, after that a link is
>>> generated for downloading. Open that in browser or do wget/curl
>>> on commandline, and it'll download a CSV with proper naming.
>>>
>>> Sample link:
>>>
>>> https://server.nikhilvj.co.in/imd_data/API/fetchData?year=2010=25.25=76.25=all
>>>
>>> You are *encouraged to scrape* : use the format above to compose your
>>> own link. (Hint: Excel formulas like: =A2&"?year=" are perfect for
>>> this)
>>>
>>> With a caveat: Given that it's not some fancy server that can take huge
>>> loads, there's a cruel and harsh rate limiter in place which will allow
>>> only 1 hit per 10 secs per IP address for anonymous users.
>>>
>>> If you want to bypass that, there's an email-otp login process given
>>> which will add a token and let you do 5 hits per second, and that won't be
>>> restricted by IP address.
>>> This would be useful in institutions where multiple users on the same
>>> network may want to fetch data at the same time.
>>> ex: a school/college hands-on session in climate data visualization
>>> where the host as usual forgets to pre-download the data
>>> So you can scrape and I can keep my server from crashing - both sides
>>> win.
>>>
>>> Source code is at https://github.com/answerquest/IMD-grid-data-work
>>> (see imd_data_api folder)
>>>
>>>
>>> --
>>> Cheers,
>>> Nikhil VJ
>>> https://nikhilvj.co.in
>>>
>>>
>>> On Wed, Oct 12, 2022 at 8:30 AM Nikhil VJ  wrote:
>>>
 Hi Pankaj,

 The place initially referred to is archival data - by definition it's
 historic and not realtime.
 What I do see on https://imdpune.gov.in/lrfindex.php is "Gridded Data
 Real Time" -> you might want to explore that.
 Over there, Daily 0.5x0.5 grid temp data links are there.
 You can select a single date there and download the file.

 I don't know right now how to open it, though. imdlib library that I've
 used - I didn't see anything about daily data.
 Here's its documentation:
 https://imdlib.readthedocs.io/en/latest/Usage.html

 I've posted a query about it in the lib's github :
 https://github.com/iamsaswata/imdlib/issues/17 , let's see if the
 author responds. Perhaps some inspection of the source code also might
 yield a way to open the daily data.

 PS: Next time do initiate a separate conversation with a different
 subject line - as here you have asked about temperature data but the
 subject reads rainfall data. Becomes less discoverable over the open
 internet or after a long period of time.

 --
 Cheers,
 Nikhil VJ
 https://nikhilvj.co.in


 On Thu, Oct 6, 2022 at 11:56 PM Pankaj pandey 
 wrote:

> Thanks Akshay for sharing the code. Actually I want to download tmax
> and tmin data at 0.5 degree not at 1 degree for 2000 to 2021. As I want to
> dowanload  gridded data  real time Download Daily Maximum Temperature
> data ( 0.5 x 0.5 degree) Binary File (Real-time). In this page only
> the select data option is available. Should i  replace year with data  in
>