[datameet] Twitter data

2020-05-20 Thread Srinivas Karuturi
Ladies and Gentlemen,

I am working on building a sentiment analysis dashboard and would like to check 
if anyone is aware of good scraping tools that can gather tweets with the 
following information based on user input of keywords and location

1. Original tweet, number of replies, number of likes, number of retweets, 
location, twitterHandle, link to the tweet
2. Retweet, number of replies, number of likes, number of retweets, location, 
twitterHandle, link to the tweet
3. Replies, number of replies, number of likes, number of retweets, location, 
twitterHandle, link to the tweet

Any information would be helpful

Regards,

Srinivas

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/BD1EA256-5761-4511-A4AD-30642160095D%40gmail.com.


Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Sajjad Anwar
I put together a scraper for this dataset a while ago. I think the data might 
be outdated but the scripts should work 
https://github.com/geohacker/toll-plazas-india


From: datameet@googlegroups.com  on behalf of Deepak 
Sharda 
Sent: Thursday, May 21, 2020 8:39:35 AM
To: datameet@googlegroups.com 
Subject: Re: [datameet] TOLL INFORMATION SYSTEM

dear souvik ,

so you share what kind of parsing tool or bot you use. we all have many queries 
like this for parsing locations from websites. it would be easier for everyone 
if you could share guidance on this matter.

thank you

On Thu, May 21, 2020 at 8:07 AM Arun Ganesh 
mailto:arungra...@gmail.com>> wrote:
Wonderful Souvik!

Have added the CSV to the gist for future reference 
https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e . Would be 
great if you could share the code to do the parsing, it would be interesting to 
parse out the rates as well.

--
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2BGKQr2zy%3DJ1qSt-MXc6B6wvn0CAYewoe5PZWiHfyB7qD%2BQZQA%40mail.gmail.com.

--
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2B4SsQDWvyjFTWLpqSJ4yWkNV8PO-p5%3De4YshSGHQay097COxw%40mail.gmail.com.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/BN8PR02MB58928D6471EA28BD90CD95B6F5B70%40BN8PR02MB5892.namprd02.prod.outlook.com.


Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Deepak Sharda
dear souvik ,

so you share what kind of parsing tool or bot you use. we all have many
queries like this for parsing locations from websites. it would be easier
for everyone if you could share guidance on this matter.

thank you

On Thu, May 21, 2020 at 8:07 AM Arun Ganesh  wrote:

> Wonderful Souvik!
>
> Have added the CSV to the gist for future reference
> https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e . Would
> be great if you could share the code to do the parsing, it would be
> interesting to parse out the rates as well.
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CA%2BGKQr2zy%3DJ1qSt-MXc6B6wvn0CAYewoe5PZWiHfyB7qD%2BQZQA%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2B4SsQDWvyjFTWLpqSJ4yWkNV8PO-p5%3De4YshSGHQay097COxw%40mail.gmail.com.


Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Arun Ganesh
Wonderful Souvik!

Have added the CSV to the gist for future reference
https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e . Would
be great if you could share the code to do the parsing, it would be
interesting to parse out the rates as well.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2BGKQr2zy%3DJ1qSt-MXc6B6wvn0CAYewoe5PZWiHfyB7qD%2BQZQA%40mail.gmail.com.


Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Souvik Sankar Mitra
I parse the json data and save the required details in csv. 

On Wednesday, May 20, 2020 at 11:40:46 PM UTC+5:30, Arun Ganesh wrote:
>
> Here's the raw json data which has a mess of encoded html. Maybe someone 
> can help parse out the required details: 
> https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e
>
> On Wed, May 20, 2020 at 2:03 PM Deepak Sharda  > wrote:
>
>> try doing web scrapping that supports locations 
>>
>> On Wed, May 20, 2020 at 8:04 PM blessing isaiah > > wrote:
>>
>>> Hi all,
>>>
>>> I would need the toll locations and the type of toll (BOT or public 
>>> funded etc) from here, 
>>> http://tis.nhai.gov.in/tollplazasonmap?language=en
>>>
>>> Can someone share it with me please. 
>>> Many thanks.
>>>
>>> Best regards,
>>> Blessing Isaiah 
>>>
>>> -- 
>>> Datameet is a community of Data Science enthusiasts in India. Know more 
>>> about us by visiting http://datameet.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to data...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/datameet/CABxwALYGs0ONwaQdEF6Cm_3eHUHVGe23fBYct2AOZ44Di0GdmQ%40mail.gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> Datameet is a community of Data Science enthusiasts in India. Know more 
>> about us by visiting http://datameet.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to data...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/datameet/CA%2B4SsQB861Gcj3gW%2BF6hkw-H0E0%3DV9yby5Req7YgHJEpqRC8fQ%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/d9811457-c57b-4e5a-8fa8-fbcc1b8f20d5%40googlegroups.com.
Tollname,Latitude,Longitude,Projecttype
Thakurtola (End of Durg Bypass),21.117914,81.122298,BOT (Toll)
Bankapur,14.90999,75.2798,Public Funded
Hirebagewadi,15.76311,74.64785,BOT (Toll)
Durg Bypass (Dhamdanaka),21.214518,81.287962,BOT (Toll)
Hattargi,16.1433,74.51696,Public Funded
Kognoli,16.5422,74.31825,Public Funded
Tundla,27.218247,78.272821,BOT (Toll)
Brahamarakotlu,12.875095,75.003849,Public Funded
Cable Stayed Naini Bridge,25.41629,81.861017,Public Funded
Bassi,25.033921,74.817676,OMT
Aroli,25.09118,75.199063,OMT
Dhaneshwar,25.072621,75.598467,OMT
Fatehpur,25.122,76.572806,OMT
Simliya,25.170089,76.130552,OMT
Omallur,11.72034,78.07338,BOT (Toll)
Nathakkarai,11.592809,78.780812,BOT (Toll)
Veeracholapuram (West),11.736464,79.010409,BOT (Toll)
Palayam (Dharmapuri),12.00643,78.08042,BOT (Toll)
Rasampalayan,11.193895,78.103219,BOT (Toll)
Nemili (Sriperumbudur),12.982155,79.968944,Public Funded
Joya,28.836176,78.420749,Public Funded
"Khalghat -MP/Maharashtra Border(Sendhwa, Jamli)",21.75667,75.156824,BOT (Toll)
Boothakudi,10.64127,78.581277,OMT
Chittampatti,9.989594,78.244086,OMT
Banglore - Neelamangla,13.090749,77.44,BOT (Toll)
Chalageri,14.556778,75.728025,Public Funded
Choundha,26.467309,78.010916,OMT
Brijghat,28.767149,78.110043,Public Funded
Usaka ( Chamari),26.036608,79.577976,Public Funded
Indore Dewas (Indore Bypass),22.822692,75.938208,BOT (Toll)
Dasna,28.684924,77.522471,Public Funded
Siwaya,29.095601,77.710995,BOT (Toll)
Daffi,25.248275,82.994017,BOT (Toll)
Mohania,25.19093,83.559544,BOT (Toll)
Sasaram,24.957125,83.954997,BOT (Toll)
Anantram,26.594931,79.268034,BOT (Toll)
Tendua (Gorakhpur Bypass),26.724015,83.274398,OMT
Barajore (Bara),26.386389,80.021485,BOT (Toll)
Bharudi,21.805381,70.663976,BOT (Toll)
Pithadiya,22.082642,70.781004,BOT (Toll)
Choryasi,21.303367,72.95418,BOT (Toll)
Anewadi,17.810312,73.971381,BOT (Toll)
Mettupatti,11.66167,78.330768,BOT (Toll)
Doddakarenahalli (Neelmangala ),13.068175,77.3605999,BOT (Toll)
Thirumandurai,11.396808,78.99356,BOT (Toll)
Paduna,24.271438,73.67379,Public Funded
Patas,18.424491,74.466193,BOT (Toll)
Thandikhui,32.607394,74.944346,Public Funded
Daroda,20.450467,78.755153,OMT
Samakhiyali,23.308555,70.49059,BOT (Toll)
Kelapur,20.019307,78.54022,Public Funded
Nandgaon Peth,20.951008,77.788359,BOT (Toll)
Mundiyar,25.208647,77.136059,OMT

Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Arun Ganesh
Here's the raw json data which has a mess of encoded html. Maybe someone
can help parse out the required details:
https://gist.github.com/planemad/fa8eb76f09ec2f95997a2b819381ea4e

On Wed, May 20, 2020 at 2:03 PM Deepak Sharda  wrote:

> try doing web scrapping that supports locations
>
> On Wed, May 20, 2020 at 8:04 PM blessing isaiah 
> wrote:
>
>> Hi all,
>>
>> I would need the toll locations and the type of toll (BOT or public
>> funded etc) from here, http://tis.nhai.gov.in/tollplazasonmap?language=en
>>
>> Can someone share it with me please.
>> Many thanks.
>>
>> Best regards,
>> Blessing Isaiah
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CABxwALYGs0ONwaQdEF6Cm_3eHUHVGe23fBYct2AOZ44Di0GdmQ%40mail.gmail.com
>> 
>> .
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CA%2B4SsQB861Gcj3gW%2BF6hkw-H0E0%3DV9yby5Req7YgHJEpqRC8fQ%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2BGKQr3xyc_fs%3DW53rame8Q0KTYDJkaAgeR9j8P%2BWjVMTxEOHA%40mail.gmail.com.


Re: [datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread Deepak Sharda
try doing web scrapping that supports locations

On Wed, May 20, 2020 at 8:04 PM blessing isaiah 
wrote:

> Hi all,
>
> I would need the toll locations and the type of toll (BOT or public funded
> etc) from here, http://tis.nhai.gov.in/tollplazasonmap?language=en
>
> Can someone share it with me please.
> Many thanks.
>
> Best regards,
> Blessing Isaiah
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CABxwALYGs0ONwaQdEF6Cm_3eHUHVGe23fBYct2AOZ44Di0GdmQ%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2B4SsQB861Gcj3gW%2BF6hkw-H0E0%3DV9yby5Req7YgHJEpqRC8fQ%40mail.gmail.com.


[datameet] WHIPS (The Wisconsin Horizontal Interpolation Program for Satellites) -- Air Pollution Feeds from NASA !!

2020-05-20 Thread Sarath Guttikunda
https://nelson.wisc.edu/sage/data-and-models/software.php

WHIPS is an open-source program designed to make satellite-derived air
quality data more usable. WHIPS interpolates level 2 satellite retrievals
onto a user-defined fixed grid, in effect creating custom-gridded level 3
satellite product. You might be interested in WHIPS if you...

Run an air quality model and want to validate it against satellite
measurements.
Plan to compare satellite data to ground-based measurements.
Need to compare multiple satellites on a common grid.
Want to boost the horizontal resolution of your satellite data by
averaging over time.

Output is flexible in resolution and projection, and can accommodate both
equi-rectangular and lambert conic conformal projections. Currently, WHIPS
can process the following data products:

OMI NO2 (NASA retrieval)
OMI NO2 (KNMI retrieval)
OMI HCHO (NASA retrieval)
OMI SO2 (NASA retrieval)
MOPITT CO (NASA retrieval)
MODIS AOD (NASA retrieval)

--
*Dr. Sarath Guttikunda*

*http://www.urbanemissions.info *

*http://www.indiaairquality.info *

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAAj%2BwWrwoS%3DnVeC%3Dc5iPMVspgtA%2BpLpBNf_yiDCfnmf%3DJ319nw%40mail.gmail.com.


[datameet] TOLL INFORMATION SYSTEM

2020-05-20 Thread blessing isaiah
Hi all,

I would need the toll locations and the type of toll (BOT or public funded
etc) from here, http://tis.nhai.gov.in/tollplazasonmap?language=en

Can someone share it with me please.
Many thanks.

Best regards,
Blessing Isaiah

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CABxwALYGs0ONwaQdEF6Cm_3eHUHVGe23fBYct2AOZ44Di0GdmQ%40mail.gmail.com.


Re: [datameet] Re: India's Weather Zones GIS file !!

2020-05-20 Thread Arun Ganesh
On Wed, May 20, 2020 at 2:17 AM Devdatta Tengshe 
wrote:

> Hi Arun,
>
> The Data seems to be in UTM 44N, i.e. EPSG:32644
>
>
Wonderful, here's the reprojected geojosn, Sarath:
https://gist.github.com/planemad/1604109e8057cb6d6822e8909468f16c

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2BGKQr2a-4GhWp92LfuWGVdLASFN4GV_Uy3TKO1pww-Q-UqFkg%40mail.gmail.com.


Re: [datameet] Re: India's Weather Zones GIS file !!

2020-05-20 Thread Devdatta Tengshe
Hi Arun,

The Data seems to be in UTM 44N, i.e. EPSG:32644

Regards,
Devdatta


On Wed, 20 May 2020 at 11:15, Arun Ganesh  wrote:

>
>
> On Tue, May 19, 2020 at 2:38 AM Sarath Guttikunda 
> wrote:
>
>> Dear Arun,
>>
>> good afternoon. These sub-divisions are specific to IMD (grouping of
>> districts) not seen in use by other departments. Power sector has their 5
>> sub-divisions (north,south,west,east, and northeast) and agro-climate zones
>> are different. The request is only to match the maps made by IMD.
>>
>>
> Interesting, after some digging, it looks like they do load a geojson of
> these divisions at runtime, but its not in the right projection. Maybe
> someone can help to fix this?
> https://mausam.imd.gov.in/imd_latest/contents/district_shapefiles/sd_boundary_rainfall.json
>
>
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CA%2BGKQr0sQHLZ1W3c95%3DyPnJW%3Dby-TrtFNeRgK3BN64XpnztQAg%40mail.gmail.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAOQOUMtp5mudGkjmb5PTRyJun4g0M7bdBP95g-Tn0mJ%2BRPpU%3DA%40mail.gmail.com.