Re: [datameet] shape files for canal network of Vishveshwaraya Canal (Left bank canal of KRS)

[Arun Ganesh]

The Vishveshwaraya Canal atleast seems to be on OSM upto Maddur. Export the
geojson here: https://overpass-turbo.eu/s/QIA

On Fri, Feb 14, 2020 at 1:49 AM Anil Dandekar  wrote:

> could someone tell me where i will get the canal network shapefiles of KRS
> dam (mysore, karnataka) especially the left bank canal which is also called
> as Vishveshwaraya Canal, any help in this regard will help me a lot for my
> research.
> thank you
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/8e4e5064-8f3e-40d9-89fe-ebbf0aa1239a%40googlegroups.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2BGKQr2h45cNL%2BjDHdkADLV%2BdAcxZ3WCtT%3DDgoKThZCP7PtLmg%40mail.gmail.com.

Re: [datameet] How to Scrape National Air Quality Data

[Thejesh GN]

No I am getting air quality.


I am downloading from the same site you mentioned

https://app.cpcbccr.com/ccr/#/caaqm-dashboard-all/caaqm-landing/caaqm-comparison-data

As of now 15 mins average of PM2.5 and PM 10.

2 days for a request.

I can go back for years. But it means 1000s of requests. Hence my note to
be soft on the site.

You can get other parameter like Benzene, Toluene etc by just modifying
this line. I was interested only in PM2.5 and PM10. Hence getting only
that. Check

https://github.com/thejeshgn/cpcbccr/blob/79374880f49a56fa1de180cfcdd13ec544eb6ffa/code/setup_pull.py#L51


Ultimately the request goes to

https://github.com/thejeshgn/cpcbccr/blob/79374880f49a56fa1de180cfcdd13ec544eb6ffa/code/pull.py#L34


Note:
The first screenshot in the blog post is for just getting site ids and not
the actual data. I think it's what is causing confusion. I will add a new
screen shot to remove that confusion.


Regards,
Thej

Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
http://thejeshgn.com

On Fri, 14 Feb, 2020, 5:36 PM Sarath Guttikunda, 
wrote:

> Dear Thej,
>
> FYI, the site you are accessing is for Air Quality Index and not Air
> Quality.
>
> As a researcher, I am interested in Air Quality Data
> Main site is this
> https://app.cpcbccr.com/ccr/#/caaqm-dashboard-all/caaqm-landing
> Use the advance search option and download the data -- cumbersome process;
> there are limitations on number of days you can request, depending on the
> time step you choose. Best is 15 mins and you take care of the weeding out;
> otherwise asking for day average, the program will give you a value
> averaging everything it has, which includes -999 and some random numbers.
>
> Alternative is openaq.org
> We keep getting blocked by once in a while from fetching and archiving.
> Only last 90 days is online and here is a post to access it
> https://observablehq.com/@olafveerman/emissions-in-wuhan
> Rest is all stored on aws s3 buckets
> https://openaq-fetches.s3.amazonaws.com/index.html (click on real time)
> There are patches of days missing when the server is blocked by cpcb from
> fetching and there is not backlog correction method in place.
>
> AQI is useful for PR purposes, but beyond that not so much. Since, this is
> a calculated value based on 24-hr averages for PM2.5 (for example), it
> missed the raw instantaneous story.
>
> With best wishes,
> Sarath
>
> --
> *Dr. Sarath Guttikunda*
>
> *http://www.urbanemissions.info *
>
> *http://www.indiaairquality.info *
>
>
> On Fri, Feb 14, 2020 at 5:11 PM Thejesh GN  wrote:
>
>> I wanted to write this how-to for a long time.  But today I did :)
>>
>> How to :
>> https://thejeshgn.com/2020/02/14/scraping-national-air-quality-data/
>> Github code: https://github.com/thejeshgn/cpcbcc
>>
>>
>> Thej
>> --
>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>> http://thejeshgn.com
>> GPG ID :  0xBFFC8DD3C06DD6B0
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CAABnYsX%3DzYCL-9g26AS69HbNz1qD5Aj%2Be%3D0YsCPRt7sdup1AbQ%40mail.gmail.com
>> 
>> .
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAAj%2BwWoRqXzcvH1FGEAQ9zCdCPOd6T1Dvd2KMeY_WkUnWv-fbA%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAABnYsWwNHYXqTJM9qf2uxn95qEic9PwbXQ9ri8fs6GKfmUusA%40mail.gmail.com.

Re: [datameet] How to Scrape National Air Quality Data

[Sarath Guttikunda]

Dear Thej,

FYI, the site you are accessing is for Air Quality Index and not Air
Quality.

As a researcher, I am interested in Air Quality Data
Main site is this
https://app.cpcbccr.com/ccr/#/caaqm-dashboard-all/caaqm-landing
Use the advance search option and download the data -- cumbersome process;
there are limitations on number of days you can request, depending on the
time step you choose. Best is 15 mins and you take care of the weeding out;
otherwise asking for day average, the program will give you a value
averaging everything it has, which includes -999 and some random numbers.

Alternative is openaq.org
We keep getting blocked by once in a while from fetching and archiving.
Only last 90 days is online and here is a post to access it
https://observablehq.com/@olafveerman/emissions-in-wuhan
Rest is all stored on aws s3 buckets
https://openaq-fetches.s3.amazonaws.com/index.html (click on real time)
There are patches of days missing when the server is blocked by cpcb from
fetching and there is not backlog correction method in place.

AQI is useful for PR purposes, but beyond that not so much. Since, this is
a calculated value based on 24-hr averages for PM2.5 (for example), it
missed the raw instantaneous story.

With best wishes,
Sarath

--
*Dr. Sarath Guttikunda*

*http://www.urbanemissions.info *

*http://www.indiaairquality.info *


On Fri, Feb 14, 2020 at 5:11 PM Thejesh GN  wrote:

> I wanted to write this how-to for a long time.  But today I did :)
>
> How to :
> https://thejeshgn.com/2020/02/14/scraping-national-air-quality-data/
> Github code: https://github.com/thejeshgn/cpcbcc
>
>
> Thej
> --
> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAABnYsX%3DzYCL-9g26AS69HbNz1qD5Aj%2Be%3D0YsCPRt7sdup1AbQ%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAAj%2BwWoRqXzcvH1FGEAQ9zCdCPOd6T1Dvd2KMeY_WkUnWv-fbA%40mail.gmail.com.

[datameet] Re: How to Scrape National Air Quality Data

[Thejesh GN]

Sorry code link is https://github.com/thejeshgn/cpcbccr

Thej
--
Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
http://thejeshgn.com
GPG ID :  0xBFFC8DD3C06DD6B0


On Fri, 14 Feb 2020 at 17:10, Thejesh GN  wrote:

> I wanted to write this how-to for a long time.  But today I did :)
>
> How to :
> https://thejeshgn.com/2020/02/14/scraping-national-air-quality-data/
> Github code: https://github.com/thejeshgn/cpcbcc
>
>
> Thej
> --
> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAABnYsVVd%3D%2BYvZU6vYG2DRQ5MG_8oGCp0HKDZjMS9WpGtHPPAA%40mail.gmail.com.

[datameet] How to Scrape National Air Quality Data

[Thejesh GN]

I wanted to write this how-to for a long time.  But today I did :)

How to :
https://thejeshgn.com/2020/02/14/scraping-national-air-quality-data/
Github code: https://github.com/thejeshgn/cpcbcc


Thej
--
Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
http://thejeshgn.com
GPG ID :  0xBFFC8DD3C06DD6B0

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAABnYsX%3DzYCL-9g26AS69HbNz1qD5Aj%2Be%3D0YsCPRt7sdup1AbQ%40mail.gmail.com.

[datameet] Re: How to get Facebook's Population Density Dataset onto Mapbox?

[Saba Mundlay]

Hi everyone!

Thanks for all your help. I managed to do it and get it onto mapbox :) 

I was looking for Delhi data, so I loaded this file 

 (map 
_43). 

I then opened in QGIS and clipped the file to the portion I needed using a 
rectangular mask + the raster clipping tool. 

After all that, I polygonized (raster to vector) the clipped file, and 
saved that as a shapefile. It loads onto mapbox perfectly. 


On Friday, January 31, 2020 at 6:02:31 PM UTC+5:30, Saba Mundlay wrote:
>
>
> Hi Friends, 
>
> I've *really* been struggling to get these datasets 
> 
>  
> onto Mapbox. Would appreciate guidance / help. The issue seems to be that 
> the dataset is a 64-bit tif, and mapbox maxes out at 8-bit tifs...
> I know that it CAN be done because of this blog 
> . 
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/0b71baed-a44e-4475-803d-5bf67a6c605d%40googlegroups.com.