[datameet] Meteorological Department (IMD) APIs

[Dilawar Singh]

Hi,

It looks like the Meteorological Department has some public APIs 
(https://mausam.imd.gov.in/imd_latest/contents/api.pdf). The document is 
silent on authorization etc.

Has anyone played with them already? If yes, is there a community resource, 
e.g., library, etc., that can access data via them?

best,
   Dilawar 

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/a9d5d623-d1d8-4b23-bb85-196f2b8e7a54n%40googlegroups.com.

Re: [datameet] Creating variable names in Devanagari

[Dilawar Singh]

I found this 
https://gis.stackexchange.com/questions/381308/how-to-change-qgis-3-default-encoding.
 



The file you are loading may have a different encoding that QGIS3. You have to 
verify this manually and make sure that QGIS3 and file encoding is the same. 

I am guessing the while QGIS3 is using utf-8, the file that you are loading may 
be utf-16 encoded. Not sure if there are automated tools that can convert 
UTF-16 files to UTF8 file while preserving unicode 
(https://superuser.com/questions/1786434/convert-utf-16-le-to-utf-8-in-windows-via-command-line).


best,

   Dilawar 






 On Mon, 04 Dec 2023 14:05:01 +0530 Sharad Lele  
wrote ---



Just to clarify/muddy the situation: I am able to enter long Devanagari text in 
the District/Jila column without any problems. It is the column/field name 
itself that I run into problems, which may have to do with the 8 characteri 
limit?




On Monday, December 4, 2023 at 1:57:04 PM UTC+5:30 Sharad Lele wrote:

Thanks, Dilawar. The article was very interesting/useful. But for a simpleton 
like me, the question then is: How do I ensure that QGIS3 in particular is 
using UTF-8 encoding every time?

Sharad


On Monday, December 4, 2023 at 1:31:28 PM UTC+5:30 Dilawar Singh wrote:

The answer is not going to be simple. 



Make sure that every time you save or open the file, the application uses 
encoding utf-8. Microsoft uses another weird encoding called utf-16 
(https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you want 
to copy-paste anything from the web or web-based tools.



Long answer: 
https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/



best,

   Dilawar 







 On Mon, 04 Dec 2023 13:24:28 +0530 Sharad Lele <> wrote ---









Hi folks, I am using QGIS and I want to create variable names in Devanagari 
(for instance जिला instead of District). If I create a new column in the 
attribute table and enter the column name in Devanagari using Microsoft Indic 
Language Tool on my computer, it seems to work (जिला is shown as the column 
name). But the moment I save the edits and reopen the attribute table, this 
gets truncated into something like जिल?. The last character is not really a 
question mark, something more complex. 



Any suggestions for making the column names appear and save properly in 
Devanagari? (The same would be true for Kannada or other scripts, I am sure).



Sharad








--

Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org

--- 

You received this message because you are subscribed to the Google Groups 
"datameet" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to .

To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com?utm_medium=email_source=footer.















--
 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google Groups 
"datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/29ffbab3-117a-4a67-8988-ae253db7e2cbn%40googlegroups.com?utm_medium=email_source=footer.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/18c3402b8ae.6e57e54a1273512.3635000473182355283%40gmail.com.

Re: [datameet] Creating variable names in Devanagari

[Dilawar Singh]

The answer is not going to be simple. 



Make sure that every time you save or open the file, the application uses 
encoding utf-8. Microsoft uses another weird encoding called utf-16 
(https://en.wikipedia.org/wiki/UTF-16).  Stay away from utf-16 it if you want 
to copy-paste anything from the web or web-based tools.



Long answer: 
https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/



best,

   Dilawar 






 On Mon, 04 Dec 2023 13:24:28 +0530 Sharad Lele  
wrote ---



Hi folks, I am using QGIS and I want to create variable names in Devanagari 
(for instance जिला instead of District). If I create a new column in the 
attribute table and enter the column name in Devanagari using Microsoft Indic 
Language Tool on my computer, it seems to work (जिला is shown as the column 
name). But the moment I save the edits and reopen the attribute table, this 
gets truncated into something like जिल?. The last character is not really a 
question mark, something more complex. 



Any suggestions for making the column names appear and save properly in 
Devanagari? (The same would be true for Kannada or other scripts, I am sure).



Sharad



--

 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org

 --- 

 You received this message because you are subscribed to the Google Groups 
"datameet" group.

 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.

 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/b7074482-2d88-4762-8765-eb8b55bee3b0n%40googlegroups.com?utm_medium=email_source=footer.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/18c33d876d0.3df668871267863.431381660657550485%40gmail.com.

[datameet] Re: [LWN] Some useful tools for binary formats

[Dilawar Singh]

The Previous link I shared is behind a paywall. I intended to send a free link 
but didn't double check during the share.  



Here it is: https://lwn.net/SubscriberLink/924133/6e25149bb7fa09ca/ 



PS: Thank Vivek Matthew for pointing out.


best,

  Dilawar






 On Wed, 01 Mar 2023 10:51:55 +0530 Dilawar Singh 
 wrote ---



https://lwn.net/Articles/924133/



best,

   Dilawar 

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/1869bcf542d.557766ac706429.1060411042507971423%40gmail.com.

[datameet] [LWN] Some useful tools for binary formats

[Dilawar Singh]

https://lwn.net/Articles/924133/



best,

   Dilawar 

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/1869b9ea550.3bb59a02697210.3220177296259052418%40gmail.com.

Re: [datameet] How I searched for the hottest villages in Kerala with Microsoft's Planetary Computer

[Dilawar Singh]

Awesome stuff! Can it be extended to all Panchayats in the country easily?



Just a nitpick: why not use "median" as an averaging strategy? 



best,

   Dilawar 


 On Fri, 24 Feb 2023 12:00:40 +0530 Shijith Kunhitty  
wrote ---



Just wanted to plug my data story on villages in Kerala and how hot they get. 



The story is at https://shijith.com/blog/kerala-panchayat-temp/ 



And the code is at https://github.com/shijithpk/hottest-panchayats-kerala 



Was thinking of using Google Earth Engine for this story, but decided to go 
with Planetary Computer since it's Python-centric and I'm more familiar with 
that language.



Pretty sure earth/environment science guys won't be happy with my methodology, 
but hopefully there's enough of a logic to it to accept the story as a basic 
analysis.



Thanks! -shijith



--

 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org

 --- 

 You received this message because you are subscribed to the Google Groups 
"datameet" group.

 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.

 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/5e8d3b97-35c4-4de2-bca4-bef27dfdfb8en%40googlegroups.com?utm_medium=email_source=footer.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/186826fd11e.84f5688a277373.8698225708209974880%40gmail.com.

Re: [datameet] Re: Data Scrape from website

[Dilawar Singh]

Did you try their API https://www.gbif.org/developer/summary ?



best,

   Dilawar 






 On Tue, 07 Feb 2023 14:35:09 +0530 Aravinth R 
 wrote ---



Hello there,I am working on a biodiversity project displaying various locations 
of wildlife species spotted in India. There is a global website called the GBIF 
(Global Biodiversity Information Facility) which uploads known locations of 
various wildlife species. The problem i am finding is i could not download the 
data for a larger area and is restricted to one species at a time. Is it 
possible to scrape the data from the website? Any information would be helpful. 
I have shared the website link here https://www.gbif.org/occurrence/map 



Regards

Aravinth R


















--
 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google Groups 
"datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/17be686e-51d2-4deb-9814-e606765c09e2n%40googlegroups.com?utm_medium=email_source=footer.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/1862b29a42b.3e215b49177117.1084882956148809571%40gmail.com.

[datameet] A public kroki server with Niolesk frontend (text to image service)

[Dilawar Singh]

Hi all,

Since we (Subconscious Compute, a seed round startup) heavily use Kroki (a
collection of tools that convert text to images), we have set up a public
server with a Niolesk frontend. You are welcome to use it. This is a public
instance. You are advised not to use it for sensitive diagrams since the
images may be cached on the server.

The URL is https://services.subcom.tech (for the frontend) and
https://kroki.subcom.tech for kroki APIs. We plan to keep it alive as long
as possible.

best,

Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Ztsny1K2AsfqRrFVe_eo07iFw_hLmMgR3s5iXu7_XOYCw%40mail.gmail.com.

Re: [datameet] Image storage, browsing and retrieval

[Dilawar Singh]

https://github.com/nextcloud/photos may just fit the bill.



- Self hosted (+/- depending on how you look at it). I run one on an AWS 
$5/month instance and it is enough for our small org right now. we don't use it 
for photos though.

- it may not be hard to link google form to Nextcloud (but Zapier doesn't have 
integration, so what do i know..).
- Have an Android app. Possibly iOS as well. You have to tag the photos. Or 
arrange them in different folders.



Nextcloud is a very popular solution and a lot of plugins. You may to browse 
through them to check if something suitable it here. Before you plan to self 
host on a dedicated server, you can try it on a shared instance. Its easy to 
install but hard to upgrade (used to break often).

 My MilesWeb shared account cost me ~ Rs. 2k per year. 



best,

   Dilawar  




 On Thu, 03 Nov 2022 10:43:16 +0530 Shivangi Desai 
 wrote ---



Hey All,



At Goonj (https://goonj.org/) we have lots of photos (from different 
projects/events/ continuous field work etc). So to manage these photos we are 
looking for photo management solutions.



We would want one in which we can add photos from the backend (using forms or 
from android applications etc) and which would have some fields associated with 
it (like boundary information, project/event information etc). So would need 
filtering and or tagging based on these parameters.



I am currently looking into google photos API for this but wanted to check if 
any of you have used any other applications or solution recommendations.



Thank You,

Regards,

Shivangi








--

 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org

 --- 

 You received this message because you are subscribed to the Google Groups 
"datameet" group.

 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.

 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAB%3DRKFtrYVbLK9jkMr9egwqJLW7xLuse%2BSjsuoy65%3DY9jdfS_A%40mail.gmail.com?utm_medium=email_source=footer.






Dilawar Singh

Cofounder & CTO, https://subcom.tech

https://www.linkedin.com/in/dilawar-singh/ / https://github.com/dilawar / 
https://orcid.org/-0002-4645-3211

Schedule a Meeting: https://subcom.zohobookings.in/#/customer/dilawar

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/1843bf22f38.539ad09a603635.6302209606396390340%40gmail.com.

Re: [datameet] Something like Holiday API (Karnataka/India)

[Dilawar Singh]

Thanks for the references, Thejesh.



For our internal use, I created a repo: https://github.com/dilawar/holidays . I 
added the list by hand. 



I'll add a frontend during the weekend.



cheers,

    Dilawar 






 On Tue, 09 Aug 2022 09:09:20 +0530 Thejesh GN  wrote 
---



Currently we don't.



May be scrape RBI holidays page?

https://m.rbi.org.in//Scripts/HolidayMatrixDisplay.aspx



And GoI calendar page



https://www.india.gov.in/calendar







On Tue, 9 Aug 2022, 09:00 Dilawar Singh, <mailto:dilawar.s.raj...@gmail.com> 
wrote:






--
 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google Groups 
"datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAABnYsVCYmrVBTkEsnqdH-jHrKR3-pZ7F3i1tgYMowKYb9u38A%40mail.gmail.com?utm_medium=email_source=footer.

Hi Datameet,



Is there a shared resources that can be used for fetching the list of holidays 
(in JSON/XML/CSV) for the current year? Something like 
https://holidayapi.com/countries/in/2022. 



May be a github repo maintained by datameet?  



I tried holiday api which is great but doesn't allow fetching the list of 
current API with free plan.



best,

    Dilawar 



--
 Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
 --- 
 You received this message because you are subscribed to the Google Groups 
"datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
email to mailto:datameet+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/8379bd89-7c8a-485e-98a7-964426dc1af9n%40googlegroups.com?utm_medium=email_source=footer.





Dilawar Singh

Cofounder & CTO, https://subcom.tech

https://www.linkedin.com/in/dilawar-singh/ / https://github.com/dilawar / 
https://orcid.org/-0002-4645-3211

Schedule a Meeting: https://subcom.zohobookings.in/#/customer/dilawar

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/18280b6dde2.2fa8f0a8289325.3415490029738110472%40gmail.com.

[datameet] Something like Holiday API (Karnataka/India)

[Dilawar Singh]

Hi Datameet,

Is there a shared resources that can be used for fetching the list of 
holidays (in JSON/XML/CSV) for the current year? Something like 
https://holidayapi.com/countries/in/2022. 

May be a github repo maintained by datameet?  

I tried holiday api which is great but doesn't allow fetching the list of 
current API with free plan.

best,
Dilawar 

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/8379bd89-7c8a-485e-98a7-964426dc1af9n%40googlegroups.com.

[datameet] Re: How to merge first and last value of each name?

[Dilawar Singh]

Look at 'difflib.SequenceMatcher' 
(https://docs.python.org/3/library/difflib.html#difflib.SequenceMatcher)  
in the Python ecosystem. Your favorite language may have something similar. 
Essentially, you are need to compute 'similarity' between two strings. 

If your prefix e.g.  AltCoin in "AlCoin Trader 20220" and "AltCoin 202132" 
are fixed and have no typos, you can solve it by regular expressions 
(https://docs.python.org/3/library/re.html). Regular Expressions (regex) 
are pretty well-supported even in GUI-based office suites.

-- Dilawar 

On Friday, February 18, 2022 at 3:24:05 PM UTC+5:30 deeptiran...@gmail.com 
wrote:

> Hi all,
>
> I have a data frame that includes yearweek (week of the year),  
> entity_name, and deposit_volume_btc (volume for each entity based on week 
> of the year). How I can merge the first and last deposit_volume for each 
> entity name?
>
> For example:
> [image: Screenshot 2022-02-14 at 4.24.57 PM.png]
> - AAX 20220 deposit_volume should be merged with AAX 202152 and similarly, 
> AltCoin Trader 20220 deposit volume should be merged with AltCoin 202132.
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/37419b29-1ead-4acc-84cd-11a07606357dn%40googlegroups.com.

Re: [datameet] Inquiry about the MEITY report on National Strategy on blockchain for govt systems

[Dilawar Singh]

Is it the one?
https://www.meity.gov.in/writereaddata/files/National_BCT_Strategy.pdf

-- Dilawar

On Tue, Dec 7, 2021 at 9:26 AM Meera Mehta  wrote:

> Dear all
>
> Had a request to share - if anyone has a copy of this report or a link to
> it
>
> MEITY report on "National Strategy on blockchain for government systems" -
> Yesterday's TOI repored on that  Link
> 
>
>
>
> Many thanks in advance
>
> Meera Mehta
>
> Professor Emeritus and Executive Director
>
> Centre for Water and Sanitation
>
> CRDF, CEPT University
>
> Ahmedabad, India
>
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CO6PR03MB6291BA1CE65E845A08EDF693D26E9%40CO6PR03MB6291.namprd03.prod.outlook.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZtiYEUnEOWvBWKiKSwcRojveWHnKEynT%3DXAGr5K8uUWjQ%40mail.gmail.com.

Re: [datameet] Measuring BJP propaganda/advertising

[Dilawar Singh]

You can get some rough numbers from https://adrindia.org/. Usually, these
are declared by political parties (so multiply by a healthy N).

There are some interesting pointers in AMA by Sophie on r/india (
https://www.reddit.com/r/IAmA/comments/oll01o/i_am_sophie_zhang_at_fb_i_worked_in_my_spare_time/).
Not sure if this is directly relevant but you can try to reach out to her.

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Tue, Nov 9, 2021 at 3:24 AM Eduardo Campillo Betancourt <
eduardocampillo2...@u.northwestern.edu> wrote:

> Hi all,
>
> I was wondering if anyone had good ideas on how to get data on being
> exposed to more BJP propaganda or advertising during elections. Is there
> perhaps data on advertising expenditures by locality or distribution of
> important local headquarters that predict a higher exposure to propaganda
> from those that live nearby? Any ideas are welcome, thank you in advance!
>
> Best,
>
> Eduardo
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/9d1ad0e0-8838-4acd-af46-bf03558a475fn%40googlegroups.com
> <https://groups.google.com/d/msgid/datameet/9d1ad0e0-8838-4acd-af46-bf03558a475fn%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZuuEhcAiDuCvOeuG-K1%3D-ZTAm88FaEW_fdLoZVLH4_oLQ%40mail.gmail.com.

Re: [datameet] Data extraction from BHUVAN website

[Dilawar Singh]

 
An Introduction to JQ: https://earthly.dev/blog/jq-select/  (worth 
bookmarking).

-- Dilawar 

On Monday, July 5, 2021 at 11:10:53 AM UTC+5:30 Puneeth Chaganti wrote:

> Hi Thej,
>
> On 04/07/21 10:51 pm, Thejesh GN wrote:
> > Puneeth thanks for the script. What one can achieve with cli (specially
> > curl and jq) always surprises me.
>
> Yes, lately I've been using jq + curl a lot more than Python, which was 
> my default choice before. :)
>
> > Is the ward geography that you got is very different from the one we
> > have here?
> > https://github.com/datameet/Municipal_Spatial_Data/tree/master/Kolkata
> > 
>
> The data gives bounding boxes for the wards, from what I understand. I 
> didn't try to visualize it, but the data has 141 wards - same as the 
> geojson linked here.
>
> I pushed the ward json data also to the gist, in case you want to take a 
> look: https://gist.github.com/4d4c326f8c70850e1ac2914d7610f0c3
>
> - Puneeth
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/e332dd12-f9e6-42f5-bbb0-9111fa890718n%40googlegroups.com.

[datameet] Developing tool to extract table from images

[Dilawar Singh]

Dear all,

I am developing a tool to extract a table from an image. It is a big
undertaking but I hope to release a beta version soon.

The input to the tool is a PNG/JPG/PDF image and output is a CSV/ODT/XLS
table.

I have some simple tables extracted from PDF. If there are formats which
govt uses often and people often need/want to digitize them, I'd like to
have some samples. I am thinking of census data, GIS data etc..

There is no plan to support multi-page tables. I can use some advice on the
OCR backend (I am using pytesseract from google for now).

best,
Dilawar

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zs9PT7CNZONjCUWM3%3D%3DiNDyfhVPg7Yhko1ALJ_Cmp25%2Bw%40mail.gmail.com.

Re: [datameet] Routing challenge: Covery every path optimally

[Dilawar Singh]

Solutions to such problems can be found in operational research literature.

Have a look at min-flow max-cut problems in graph theory which are suited
to solve this kind of optimization problem (
https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.flow.minimum_cut.html).
This problem can be converted into an instance of this problem after
suitable transformations. Another approach could be finding a balanced
partition of a graph (gomory-hu etc).

I don't see a computational challenge here since the size of the problem is
likely to be a few hundred places and thousands on possible routes. Maybe a
brute force algo with do the job as well.

PS: Talk is cheap! If you show me the data, I can show you the code.

Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Tue, May 11, 2021 at 6:14 PM Ujaval Gandhi 
wrote:

> A colleague of mine solved a similar problem using the OpenRouteService
> API. They used k-means clusters to determine the initial distribution of
> points between surveyors and the optimization API
> <https://openrouteservice.org/dev/#/api-docs/optimization/post> for
> solving optimal routing between the points.
>
> Another less glamorous but maybe a more practical solution: Overlay a grid
> and count the length of roads inside each grid. Assign grids to each
> surveyor. You can add distance from starting point in the calculation as
> well. I have run field operations before, and a grid-based approach is
> usually more manageable than a complex 'start here and walk the streets in
> this order'.
>
> Good luck!
> [image: Logo] <https://spatialthoughts.com/>
> Ujaval Gandhi
> Spatial Thoughts
> mobile: +91-8095684687
> email: uja...@spatialthoughts.com
> [image: LinkedIn icon] <https://www.linkedin.com/in/spatialthoughts/>  [image:
> Twitter icon] <https://twitter.com/spatialthoughts>
>
>
>
> On Tue, May 11, 2021 at 9:09 AM Nikhil VJ  wrote:
>
>> Hi All,
>>
>> Wishing everyone good health, stability and pragmatism in these times.
>> I came across a certain technical problem statement pertaining to ground
>> survey planning in a target area:
>>
>> Given X ground surveyors,
>> Create X routes that start from one location,
>> Cover all the existing roads and pedestrian pathways in the target area
>> (obtained from OpenStreetMap data),
>> Such that each path is walked over at least once.
>> Balance the distance amongst the routes so that no one gets the brunt of
>> the tasks.
>>
>> Variant 1: Multiple starting locations allowed.
>>
>> Reaching out to check if anyone has experience working this out? It seems
>> like a common/recurring challenge that can use a common solution.
>>
>> I'm checking out OSMNX, but not finding a usable example yet over there.
>>
>> One idea: Plot a point at say every 50 meters along all the paths.
>> Inspect and adjust manually at intersections etc. Then run a travelling
>> salesman type algorithm on it to ensure that each point has been covered at
>> least once.
>>
>> Another idea: Create a user interface to assist a person to work out the
>> solution manually - make selections, plot the routes and see the result,
>> tweak the selections and try again. Less glamorous but possibly more
>> effective than chasing behind exotic algorithms.
>>
>> One base dataset required for such problems is: distance matrix. Another:
>> way to map on-road route between any two points, lots of times. I've got
>> those sorted out using OSRM, so no worries on that front.
>>
>> Please forward this to colleges / students that might be looking for such
>> problem statements to take up. I can setup an official internship if
>> required.
>>
>> --
>> Cheers,
>> Nikhil VJ
>> https://nikhilvj.co.in
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CAH7jeuPsR63qjLvXzHw6iOkRCotXhw5jX175gKVJcks3sLPN9Q%40mail.gmail.com
>> <https://groups.google.com/d/msgid/datameet/CAH7jeuPsR63qjLvXzHw6iOkRCotXhw5jX175gKVJcks3sLPN9Q%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
> --
>

Re: [datameet] Indian States centroids- shape file

[Dilawar Singh]

Have a look: https://github.com/dilawar/map-india-center

best,

Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Wed, Mar 3, 2021 at 12:32 PM Rajesvari Parasa 
wrote:

> Hey,
> It should be very easy to do in QGIS. Here
> <https://gis.stackexchange.com/questions/45243/how-to-determine-the-centroid-of-polygons>.
> GIS Stackexchange should be your go to.
>
> And since you're here assuming you already know where to find the states
> polygon files: Datameet maps repo. <https://github.com/datameet/maps>
>
> On Wed, Mar 3, 2021 at 11:49 AM Haripriya  wrote:
>
>> Hey guys,
>>
>> I am looking for a shapefile of all the centroids of the Indian states.
>> It would be great if I could find some guidance since I am new to GIS.
>> Thank you!
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/a45ab208-b016-42c4-8414-3cdf306ed582n%40googlegroups.com
>> <https://groups.google.com/d/msgid/datameet/a45ab208-b016-42c4-8414-3cdf306ed582n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAEKTdxrrW%3DDvSJpJto5CyWhWNvJwnB2tpHg8L6VF-4Qr1B-9cw%40mail.gmail.com
> <https://groups.google.com/d/msgid/datameet/CAEKTdxrrW%3DDvSJpJto5CyWhWNvJwnB2tpHg8L6VF-4Qr1B-9cw%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZvNdj_bSS9dkjiQC8SNsKMG3sQpgDOybv4zxObFbe_CgA%40mail.gmail.com.

Re: [datameet] Chennai becomes the first Indian city to have a comprehensive automatic groundwater monitoring system.

[Dilawar Singh]

Great.

The EXCEL format can use some more work. Currently, it is not very machine
friendly (probably an HTML table is exported directly to both PDF and
EXCEL). A plain text CSV format would be very helpful.

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Fri, Feb 5, 2021 at 6:03 PM Vaishnavi Jayakumar <
jayakumar.vaishn...@gmail.com> wrote:

> Http://threadreaderapp.com/thread/1357374096208580608.html
>
> Chennai becomes the first Indian city to have a comprehensive automatic
> groundwater monitoring system.200 Automatic GW level recorders (1 in every
> ward), 20 Rain gauges have been installed (1 each in 15 zones & 5 lakes) by
> @CHN_Metro_Water. A video explaining how it works -
>
> One can select his location out of the 200 locations and can have a
> real-time record of Ground water levels in his locality.
>
> Also reports of longitudinal GW data of a particular location over the
> selected time period can be downloaded. Historical GW data dating back to
> 1987 can also be retrieved.
>
> Similarly, real-time rainfall data and historical data from 1965 & daily
> lake levels can be accessed from the portal. In summary, this is a
> milestone in the country in quantifying the water cycle and also making
> information accessible to the citizens. @CGWB_CHQ @JalShaktiAbhyan
>
> Real time data of Ground water, rainfall & lake levels can be accessed by
> anyone in the comprehensive portal in @CHN_Metro_Water website by following
> this link - 
> *http://122.183.188.248:8080/CMWSSB-web/onlineMonitoringSystem/waterLevel
> <http://122.183.188.248:8080/CMWSSB-web/onlineMonitoringSystem/waterLevel>*
>
> Suggestions for improvement are welcome from one and all.
>
> https://twitter.com/prabhusean7/status/1357374096208580608?s=19
>
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAN7%2Bzvqt_XM9ByjwbakiQK%3DVudwQB0QR-chC%2BCjS4ikptz29mg%40mail.gmail.com
> <https://groups.google.com/d/msgid/datameet/CAN7%2Bzvqt_XM9ByjwbakiQK%3DVudwQB0QR-chC%2BCjS4ikptz29mg%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZtqGyN6ZJfXgkdRQrmzicxbimkFfKNtGZD%2BsYAD5umFhg%40mail.gmail.com.

Re: [datameet] Trouble reading Indian Districts Database's files

[Dilawar Singh]

Hi Eduardo,

Structured data will be useful for anyone visiting this thread. You can
send them to me (I'll add them to the Github link); or you can also post
them to a public server and post a link here.

best,
Dilawar

On Wed, Feb 3, 2021 at 2:40 AM Eduardo Campillo Betancourt <
eduardocampillo2...@u.northwestern.edu> wrote:

>
> Hi Dilawar,
>
> Thank you so much! The SAS files were very useful in understanding the
> structure. They're surprisingly not mangled, just very strangely arranged!
> Let me know if you need them in .dta and I can share the files once I'm
> done cleaning them.
>
> Best,
> Eduardo
> On Sunday, January 31, 2021 at 10:21:03 PM UTC-6 dilawar@gmail.com
> wrote:
>
>> Found one:
>> https://web.archive.org/web/20130314062432/http://www.bsos.umd.edu/socy/vanneman/districts/files/index.html
>> . SAS files are available in this snapshot.
>>
>> --
>> Dilawar Singh, Ph.D.
>> LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/>
>> ORCID <https://orcid.org/-0002-4645-3211> Github
>> <https://github.com/dilawar>
>>
>>
>> On Mon, Feb 1, 2021 at 9:45 AM Dilawar Singh 
>> wrote:
>>
>>> These files are hopelessly mangled. I've extracted the data and put them
>>> here
>>> https://github.com/dilawar/data/tree/master/IndianDistrictDatabase/original
>>> . Most entries have 5 columns but some are one column and two columns.
>>>
>>> You can search for this site snapshots on the internet archive and if
>>> you are lucky, some old snapshots might have the currently missing SAS
>>> files. The last time I tried, their search engine was having issues:
>>>
>>>
>>> https://archive.org/search.php?query=http%3A%2F%2Fvanneman.umd.edu%2Fdistricts%2Ffiles%2Findex.html
>>>
>>> --
>>> Dilawar Singh, Ph.D.
>>> LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/>
>>> ORCID <https://orcid.org/-0002-4645-3211> Github
>>> <https://github.com/dilawar>
>>>
>>>
>>> On Mon, Feb 1, 2021 at 8:40 AM Eduardo Campillo Betancourt <
>>> eduardoca...@u.northwestern.edu> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Have any of you successfully read the .data files in the Indian
>>>> Districts Database (http://vanneman.umd.edu/districts/files/index.html)?
>>>> I tried reading them in R as ASCII and elsewhere as text files or tab
>>>> separated files and all I got was a single column of about 150K rows with
>>>> what looked like the data but with no sense of which column corresponds to
>>>> which variable. Also, the SAS command files are not available (links
>>>> broken).
>>>>
>>>> Thank you in advance!
>>>>
>>>> Best,
>>>> Eduardo
>>>>
>>>> --
>>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>>> about us by visiting http://datameet.org
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "datameet" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to datameet+u...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/datameet/5566c50b-9009-4bd9-a0a5-22b0247e2d6bn%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/datameet/5566c50b-9009-4bd9-a0a5-22b0247e2d6bn%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/cb050fb1-3b73-45e5-9370-988342d8c342n%40googlegroups.com
> <https://groups.google.com/d/msgid/datameet/cb050fb1-3b73-45e5-9370-988342d8c342n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZvNmH2MDoiZJRJow6978P0Dtm-dFiV-XQ7nGF2G0hmmug%40mail.gmail.com.

Re: [datameet] Trouble reading Indian Districts Database's files

[Dilawar Singh]

Found one:
https://web.archive.org/web/20130314062432/http://www.bsos.umd.edu/socy/vanneman/districts/files/index.html
. SAS files are available in this snapshot.

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Mon, Feb 1, 2021 at 9:45 AM Dilawar Singh 
wrote:

> These files are hopelessly mangled. I've extracted the data and put them
> here
> https://github.com/dilawar/data/tree/master/IndianDistrictDatabase/original
> . Most entries have 5 columns but some are one column and two columns.
>
> You can search for this site snapshots on the internet archive and if you
> are lucky, some old snapshots might have the currently missing SAS files.
> The last time I tried, their search engine was having issues:
>
>
> https://archive.org/search.php?query=http%3A%2F%2Fvanneman.umd.edu%2Fdistricts%2Ffiles%2Findex.html
>
> --
> Dilawar Singh, Ph.D.
> LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
> <https://orcid.org/-0002-4645-3211> Github
> <https://github.com/dilawar>
>
>
> On Mon, Feb 1, 2021 at 8:40 AM Eduardo Campillo Betancourt <
> eduardocampillo2...@u.northwestern.edu> wrote:
>
>> Hi all,
>>
>> Have any of you successfully read the .data files in the Indian Districts
>> Database (http://vanneman.umd.edu/districts/files/index.html)? I tried
>> reading them in R as ASCII and elsewhere as text files or tab separated
>> files and all I got was a single column of about 150K rows with what looked
>> like the data but with no sense of which column corresponds to which
>> variable. Also, the SAS command files are not available (links broken).
>>
>> Thank you in advance!
>>
>> Best,
>> Eduardo
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/5566c50b-9009-4bd9-a0a5-22b0247e2d6bn%40googlegroups.com
>> <https://groups.google.com/d/msgid/datameet/5566c50b-9009-4bd9-a0a5-22b0247e2d6bn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZtoyZyTLfOQegYfQP3wr8qknA4%3Dh6YQaBuUqP1MddVEQg%40mail.gmail.com.

Re: [datameet] Trouble reading Indian Districts Database's files

[Dilawar Singh]

These files are hopelessly mangled. I've extracted the data and put them
here
https://github.com/dilawar/data/tree/master/IndianDistrictDatabase/original
. Most entries have 5 columns but some are one column and two columns.

You can search for this site snapshots on the internet archive and if you
are lucky, some old snapshots might have the currently missing SAS files.
The last time I tried, their search engine was having issues:

https://archive.org/search.php?query=http%3A%2F%2Fvanneman.umd.edu%2Fdistricts%2Ffiles%2Findex.html

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Mon, Feb 1, 2021 at 8:40 AM Eduardo Campillo Betancourt <
eduardocampillo2...@u.northwestern.edu> wrote:

> Hi all,
>
> Have any of you successfully read the .data files in the Indian Districts
> Database (http://vanneman.umd.edu/districts/files/index.html)? I tried
> reading them in R as ASCII and elsewhere as text files or tab separated
> files and all I got was a single column of about 150K rows with what looked
> like the data but with no sense of which column corresponds to which
> variable. Also, the SAS command files are not available (links broken).
>
> Thank you in advance!
>
> Best,
> Eduardo
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/5566c50b-9009-4bd9-a0a5-22b0247e2d6bn%40googlegroups.com
> <https://groups.google.com/d/msgid/datameet/5566c50b-9009-4bd9-a0a5-22b0247e2d6bn%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zu1QzqJ5hD4TrULhrsnNhScVE3hMm%2BqbR-n%2BHUqvK0U-g%40mail.gmail.com.

Re: [datameet] The State of FOSS in India | Report Launch | 12th-15th Janurary 2021

[Dilawar Singh]

There is some relevant data in the FOSS report published recently.

https://www.linuxfoundation.org/wp-content/uploads/2020FOSSContributorSurveyReport_121020.pdf

[image: Firefox_Screenshot_2021-01-13T11-49-48.288Z.png]
A healthy percentage of developers are paid to contribute to FOSS in the
world (roughly 50%). India has the lowest percentage. The number of
respondents is pretty low here but it goes well with the anecdotal
evidences.

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Wed, Jan 13, 2021 at 4:47 PM gggod...@gmail.com 
wrote:

> Please access report and relevant sources at: https://state-of-foss.in/
>
> Thanks!
>
> Regards,
> Gaurav
>
> On Wednesday, 13 January 2021 at 08:58:25 UTC+5:30 anivar@gmail.com
> wrote:
>
>> On Mon, Jan 11, 2021 at 10:38 AM gggod...@gmail.com 
>> wrote:
>>
>>>
>>>
>>> <https://comms.civicdatalab.in/r/d5ce1d52c6c8eee58d2b5e446?ct=YTo1OntzOjY6InNvdXJjZSI7YToyOntpOjA7czo1OiJlbWFpbCI7aToxO2k6Mjt9czo1OiJlbWFpbCI7aToyO3M6NDoic3RhdCI7czoyMjoiNWZmNmM2YTFlNmUxNDg4NjEwMjgzMyI7czo0OiJsZWFkIjtzOjQ6IjQwMjIiO3M6NzoiY2hhbm5lbCI7YToxOntzOjU6ImVtYWlsIjtpOjI7fX0%3D;>
>>> More than 85% of India’s Internet runs on FOSS.
>>>
>>
>> Source/ Data for this?
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/cb4c7a8c-4014-4a30-bb29-21bac1b68a39n%40googlegroups.com
> <https://groups.google.com/d/msgid/datameet/cb4c7a8c-4014-4a30-bb29-21bac1b68a39n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zv7JR9Qe5jU%3DdH-Od9uOzeywoDUeDVjz7iTy9pVk-EkCw%40mail.gmail.com.

Re: [datameet] WikiData and simlar platforms

[Dilawar Singh]

It would be nice to have a look at available user-facing tools for Wikidata
and Datacommons. I believe the DataCommons has more bells and whistles
(user can interact with data in the browser) while wikidata lacks such
interactive tools. May be such tools are available from third parties.

Nonetheless, my vote is for WikiData as the official source, and if
possible mirror it to DataCommons so people can explore it easily .

--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Thu, Dec 24, 2020 at 10:23 PM Arun Ganesh  wrote:

>
>
> On Thu, Dec 24, 2020 at 12:19 AM Thejesh GN  wrote:
>
>> Hi All,
>>
>> Last few months, I have been experimenting with WikiData, DataCommons,
>> Semantic Web <https://thejeshgn.com/tag/semantic-web/>, Knowledge graph,
>> etc.
>>
>>
>> Over time I am planning to move some data slowly to WikiData. I will keep
>> this post updated. The learning curve is a little steep. But I think
>> WikiData and similar platforms seem to natural allies to what we do here,
>> just like OSM. So as a community, I think it's a good investment of time.
>>
>> What do you guys think?
>>
>>
> 100% support!
>
> Supporting and contributing back to wikidata is a natural fit. All the
> data is public domain CC-0 and really is the master of all open datasets.
>
> For those who have no background on Wikidata, it's a sibling project of
> Wikipedia with a goal to semantically organize all knowledge on earth as an
> open queryable datastore. Every concept/topic has a unique stable
> identifier and consists of statements that define it, eg India =
> https://www.wikidata.org/wiki/Q668 . Also check out this nice overview of
> Wikidata+OSM by Naveen Francis
> https://wiki.openstreetmap.org/w/images/0/0a/Wikidata%2BOpenStreetMap.pdf
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CA%2BGKQr0jQ7f394Dn9hGMQvhZkTAZ_47BYpXB4BErZ%2BVavMTpSw%40mail.gmail.com
> <https://groups.google.com/d/msgid/datameet/CA%2BGKQr0jQ7f394Dn9hGMQvhZkTAZ_47BYpXB4BErZ%2BVavMTpSw%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZsNvAChzsd6CZc2225QDn71DzU9_kFn_Jq3kj6cQLPaRg%40mail.gmail.com.

Re: [datameet] Need help to run .mvt files

[Dilawar Singh]

On a bit more exploration.

Here is a script that can read these file and emit JSON:
https://raw.githubusercontent.com/dilawar/Scripts/7903397d8f5134ca08c03f2767eeb19db7a0f902/mvt2json.py

Example:

$ python3 mvt2json.py ~/Downloads/censuswardboundary_719_456.mvt

{'censuswardboundary': {'extent': 4096, 'version': 2, 'features':
[{'geometry': {'type': 'Polygon', 'coordinates': [[[131, 3552], [172,
3525], [187, 3518], [204, 3496], [220, 3497], [231, 3500], [275,
3503], [317, 3502], [318, 3498], [322, 3497], [326, 3489], [329,
3488], [323, 3479], [328, 3468], [327, 3459], [338, 3461], [345,
3459], [355, 3453], [391, 3421], [437, 3370], [447, 3354], [470,
3331], [485, 3318], [501, 3309], [539, 3297], [556, 3293], [572,
3288], [570, 3277], [561, 3254], [558, 3237], [555, 3233], [554,
3224], [547, 3204], [541, 3204], [545, 3202], [544, 3196], [535,
3177], [531, 3175], [526, 3175], [527, 3174], [528, 3170], [528,
3165], [522, 3152], [511, 3134], [507, 3130], [503, 3128], [494,
3115], [431, 3065], [410, 3042], [400, 3042], [405, 3037], [400,
3041], [399, 3040], [402, 3035], [381, 3007], [362, 2965], [351,
2981], [333, 2992], [308, 3003], [285, 3015], [277, 3018], [246,
3022], [239, 3030], [237, 3042], [131, 3039], [114, 3035], [110,
3030], [105, 3029], [105, 3024], [104, 3024], [102, 3018], [91, 3012],
[95, 2980], [90, 2980], [87, 2976], [81, 2976], [79, 2974], [80,
2966], [90, 2959], [72, 2941], [70, 2941], [71, 2947], [61, 2957],
[61, 2964], [62, 2969], [60, 2972], [51, 2977], [45, 2986], [38,
2988], [33, 2996], [31, 3002], [30, 3017], [31, 3024], [29, 3031],
[32, 3042], [39, 3052], [44, 3055], [47, 3065], [56, 3074], [63,
3077], [63, 3079], [60, 3082], [65, 3084], [67, 3094], [75, 3098],
[80, 3106], [68, 3166], [54, 3179], [49, 3199], [61, 3196], [70,
3199], [76, 3205], [85, 3223], [86, 3231], [82, 3241], [78, 3238],
[75, 3241], [68, 3241], [78, 3...



Will output the JSON to console which you can redirect to a file. Attached
is one of your files converted to JSON file.

This script depends on https://github.com/tilezen/mapbox-vector-tile. MVT
specification is here
https://github.com/mapbox/vector-tile-spec/tree/master/2.1 (once you know
it searching for tools is easy).

This is not the most convenient solution but if you ask for a better tool,
mention this on the gis.stackexchange.com site. It's good to do your own
homework :).

best,
Dilawar
--
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Wed, Dec 16, 2020 at 2:41 AM Dilawar Singh 
wrote:

> Reference: https://gdal.org/drivers/vector/mvt.html
>
> It doesn't look like GDAL is capable of converting MVT with 3d values to
> GeoJSON. Most of your GIS software uses gdal behind the scene for
> conversion.
>
> [dilawars@chutki Downloads]$ ogr2ogr -f GeoJSON a.geojson 
> censuswardboundary_719_456.mvt
> ERROR 1: Layer 'censuswardboundary_label' does not already exist in the 
> output dataset, and cannot be created by the output driver.
> ERROR 1: Terminating translation prematurely after failed
> translation of layer censuswardboundary_label (use -skipfailures to skip 
> errors)
>
> Try https://gis.stackexchange.com/ for better luck.
>
> best,
>Dilawar
>
> On Wed, Dec 16, 2020 at 12:04 AM Arun Ganesh  wrote:
>
>> It’s hard to know what’s happening without looking at the actual mvt.
>> Where did you get it from?
>>
>> On Tue, Dec 15, 2020 at 6:25 AM Vijay Sapkal 
>> wrote:
>>
>>> Hello,
>>> When I am searching EPSG: 4326 and selecting it to run file. It is not
>>> showing any result.
>>> I am not able to see any point/line/polygon. Please help.
>>>
>>> Thanks,
>>> Vijay
>>>
>>> On Tue, Dec 15, 2020 at 4:25 PM Souvik Sankar Mitra <
>>> souviksankar2...@gmail.com> wrote:
>>>
>>>> I think EPSG: 4326 is a good option. Means WGS84
>>>>
>>>> On Tue, Dec 15, 2020 at 4:03 PM Vijay Sapkal 
>>>> wrote:
>>>>
>>>>> Hey, while running this file in QGIS, it is asking me to select CRS
>>>>> (coordinate reference system), which one I should select for this file?
>>>>>
>>>>> On Tue, Dec 15, 2020 at 3:49 PM Vijay Sapkal 
>>>>> wrote:
>>>>>
>>>>>> Thank you, l'll try and get back to you.
>>>>>>
>>>>>> On Tue, Dec 15, 2020 at 3:36 PM Souvik Sankar Mitra <
>>>>>> souviksankar2...@gmail.com> wrote:
>>>>>>
>>>>>>> Open it in QGIS. Then export it in other formats.
>>>>>>>
>>>>>>> On Tue, Dec 15, 2020 at 3:29 PM Vijay Sapkal

Re: [datameet] Need help to run .mvt files

[Dilawar Singh]

Reference: https://gdal.org/drivers/vector/mvt.html

It doesn't look like GDAL is capable of converting MVT with 3d values to
GeoJSON. Most of your GIS software uses gdal behind the scene for
conversion.

[dilawars@chutki Downloads]$ ogr2ogr -f GeoJSON a.geojson
censuswardboundary_719_456.mvt
ERROR 1: Layer 'censuswardboundary_label' does not already exist in
the output dataset, and cannot be created by the output driver.
ERROR 1: Terminating translation prematurely after failed
translation of layer censuswardboundary_label (use -skipfailures to skip errors)

Try https://gis.stackexchange.com/ for better luck.

best,
   Dilawar

On Wed, Dec 16, 2020 at 12:04 AM Arun Ganesh  wrote:

> It’s hard to know what’s happening without looking at the actual mvt.
> Where did you get it from?
>
> On Tue, Dec 15, 2020 at 6:25 AM Vijay Sapkal  wrote:
>
>> Hello,
>> When I am searching EPSG: 4326 and selecting it to run file. It is not
>> showing any result.
>> I am not able to see any point/line/polygon. Please help.
>>
>> Thanks,
>> Vijay
>>
>> On Tue, Dec 15, 2020 at 4:25 PM Souvik Sankar Mitra <
>> souviksankar2...@gmail.com> wrote:
>>
>>> I think EPSG: 4326 is a good option. Means WGS84
>>>
>>> On Tue, Dec 15, 2020 at 4:03 PM Vijay Sapkal 
>>> wrote:
>>>
 Hey, while running this file in QGIS, it is asking me to select CRS
 (coordinate reference system), which one I should select for this file?

 On Tue, Dec 15, 2020 at 3:49 PM Vijay Sapkal 
 wrote:

> Thank you, l'll try and get back to you.
>
> On Tue, Dec 15, 2020 at 3:36 PM Souvik Sankar Mitra <
> souviksankar2...@gmail.com> wrote:
>
>> Open it in QGIS. Then export it in other formats.
>>
>> On Tue, Dec 15, 2020 at 3:29 PM Vijay Sapkal 
>> wrote:
>>
>>> Hello team,
>>>
>>> I need help to run .mvt file. Is it possible for anyone to do it
>>> quickly for me?
>>> Waiting for your reply,
>>>
>>> Thanks,
>>> Vijay
>>>
>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know
>>> more about us by visiting http://datameet.org
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to datameet+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/datameet/CA%2BzgrCRqt1oYTuof%2BnhMsVsSX%3DEVKHdctO-k%2BmGahH2fU86CeA%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know
>> more about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google
>> Groups "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CAM88csWbcdU5dbm8ttP%3D6%3Dx5yKMKzwCwjue_dR3f-F49_LiAfw%40mail.gmail.com
>> 
>> .
>>
> --
 Datameet is a community of Data Science enthusiasts in India. Know more
 about us by visiting http://datameet.org
 ---
 You received this message because you are subscribed to the Google
 Groups "datameet" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to datameet+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/datameet/CA%2BzgrCS-RwhWws%2BJSGLas%3DhyDQUfZkcxeM40eji5VGk05_S-Nw%40mail.gmail.com
 
 .

>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>> about us by visiting http://datameet.org
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to datameet+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/datameet/CAM88csXSZ5pjEXgVE0MMgANzXGQXXt1d0SvHi_%3D%2B7GtwXvJ%2BWg%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting 

Re: [datameet] Discord

[Dilawar Singh]

Credit: xkcd.

On Sat, Dec 5, 2020, 21:11 Nikhil VJ  wrote:

> Are there distributed alternatives to emails which has better support of
> structured knowledge management?
>
>
> *Discourse* forum? https://www.discourse.org/
> I've seen some pretty good networking happening on this - ODK Forum
> <https://forum.getodk.org/> (for mobile based survey tools) is one
> example. It's open source with lot of open integrations
> <https://www.discourse.org/integrations> and active coding community
> working on it. I've set a test one up recently without too much trouble on
> a 4GB hetzner server that costs under rs.1k/mo. You need to map your domain
> to it and have smtp email working  - I was able to do it with a bigrock.in
> domain. There would be scaling up challenges of course, but can be handled.
> (more usage = more capability to ramp up). Search works well.
>
>
> --
> Cheers,
> Nikhil VJ
> https://nikhilvj.co.in
>
>
> On Sat, Dec 5, 2020 at 1:13 AM Dilawar Singh 
> wrote:
>
>> Slack is not an ideal platform for collecting open knowledge or data. In
>> addition to what Thajesh has already mentioned, slack is a vendor lock-in
>> which should be a big NO. Moreover, If I am not wrong, they don't let you
>> browse post 10k messages in the free plan. Personally, I can live without
>> one less account.
>>
>> Not to mention that there is a non-zero probability they will be around
>> after 5 or 10 years. The same argument applies to any centralized platform.
>> Are there distributed alternatives to emails which has better support of
>> structured knowledge management?
>>
>> best,
>> Dilawar Singh, Ph.D.
>> LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/>
>> ORCID <https://orcid.org/-0002-4645-3211> Github
>> <https://github.com/dilawar>
>>
>>
>> On Sat, Dec 5, 2020 at 12:35 AM shaily sparsha 
>> wrote:
>>
>>> Agreed for slack.
>>>
>>> On Fri, Dec 4, 2020 at 1:32 AM BNP Analytics 
>>> wrote:
>>>
>>>> Agreed.
>>>> A slack or discord channel will be good.
>>>> Mainly as they may support better code sharing.
>>>>
>>>>
>>>> On Tue, Dec 1, 2020 at 5:27 AM Akshay Aditya 
>>>> wrote:
>>>>
>>>>> Just a message to everyone.
>>>>> Instead of the usual format of emailing on the group or commenting on
>>>>> the groups. Should we consider doing this on discord?, we would be able to
>>>>> share information more quickly and threads would be easier to follow.
>>>>> Sometimes I’m looking for something and it’s a little hard to find them
>>>>> because of the email format.
>>>>>
>>>>> Let me know what you all think.
>>>>>
>>>>> --
>>>>> Datameet is a community of Data Science enthusiasts in India. Know
>>>>> more about us by visiting http://datameet.org
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "datameet" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to datameet+unsubscr...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/datameet/250EDAA2-4F8B-4498-A481-6F141700A5A5%40gmail.com
>>>>> .
>>>>>
>>>> --
>>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>>> about us by visiting http://datameet.org
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "datameet" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to datameet+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/datameet/CAFJrGLGwg0zRONi%2BoEeLFz96DTJOSsw1ZbfTf2qyfr%3DV-a2j3Q%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/datameet/CAFJrGLGwg0zRONi%2BoEeLFz96DTJOSsw1ZbfTf2qyfr%3DV-a2j3Q%40mail.gmail.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>>
>>>
>>> Regards,
>>> SHAILY SPARSHA
>>>
>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>> about us by visiting http://datameet.org
>>> ---
>>> You received this message because you ar

Re: [datameet] Discord

[Dilawar Singh]

Slack is not an ideal platform for collecting open knowledge or data. In
addition to what Thajesh has already mentioned, slack is a vendor lock-in
which should be a big NO. Moreover, If I am not wrong, they don't let you
browse post 10k messages in the free plan. Personally, I can live without
one less account.

Not to mention that there is a non-zero probability they will be around
after 5 or 10 years. The same argument applies to any centralized platform.
Are there distributed alternatives to emails which has better support of
structured knowledge management?

best,
Dilawar Singh, Ph.D.
LinkedIn <https://www.linkedin.com/in/dilawar-singh-ph-d-44b81b194/> ORCID
<https://orcid.org/-0002-4645-3211> Github <https://github.com/dilawar>


On Sat, Dec 5, 2020 at 12:35 AM shaily sparsha 
wrote:

> Agreed for slack.
>
> On Fri, Dec 4, 2020 at 1:32 AM BNP Analytics 
> wrote:
>
>> Agreed.
>> A slack or discord channel will be good.
>> Mainly as they may support better code sharing.
>>
>>
>> On Tue, Dec 1, 2020 at 5:27 AM Akshay Aditya 
>> wrote:
>>
>>> Just a message to everyone.
>>> Instead of the usual format of emailing on the group or commenting on
>>> the groups. Should we consider doing this on discord?, we would be able to
>>> share information more quickly and threads would be easier to follow.
>>> Sometimes I’m looking for something and it’s a little hard to find them
>>> because of the email format.
>>>
>>> Let me know what you all think.
>>>
>>> --
>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>> about us by visiting http://datameet.org
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to datameet+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/datameet/250EDAA2-4F8B-4498-A481-6F141700A5A5%40gmail.com
>>> .
>>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CAFJrGLGwg0zRONi%2BoEeLFz96DTJOSsw1ZbfTf2qyfr%3DV-a2j3Q%40mail.gmail.com
>> <https://groups.google.com/d/msgid/datameet/CAFJrGLGwg0zRONi%2BoEeLFz96DTJOSsw1ZbfTf2qyfr%3DV-a2j3Q%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
>
>
> Regards,
> SHAILY SPARSHA
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CAHnqzP0K0sTiS60GU5G5EDY21-_SZrRWY_9NsQfsvfQndAeZUw%40mail.gmail.com
> <https://groups.google.com/d/msgid/datameet/CAHnqzP0K0sTiS60GU5G5EDY21-_SZrRWY_9NsQfsvfQndAeZUw%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZtQbeqmjosoMC%3DQ6YoQvM%3Damm8eSWV2Rwk9BgbSbWwCuw%40mail.gmail.com.

Re: [datameet] TEXT TO ASCII

[Dilawar Singh]

> I have a set of files with X, Y, Z Values (Lidar Data) in .txt files. I
need to convert it to ASCII format,

This line is very confusing. `.txt` extensions usually means ASCII (or
unicode). It is usually a good idea to attach a sample of the file to
reduce the guesswork.

You may also want to explore https://stackoverflow.com/ .

best,
Dilawar

On Sun, Sep 6, 2020 at 1:04 PM blessing isaiah 
wrote:

> Hi all,
>
> I have a set of files with X, Y, Z Values (Lidar Data) in .txt files. I
> need to convert it to ASCII format, Kindly help me to go about it. R/Python
> scripting or any other method - model builders in QGIS/ ArcMAP or even FME
> will help.
>
> Any idea or leads is appreciated. Thank you in advance.
>
> Regards,
> Blessing
>
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CABxwALbB11NgD3kv-NG1J5ZasSjKCTzvAYUjWCbrLEF1qPOkmg%40mail.gmail.com
> 
> .
>


-- 
Dilawar
NCBS Bangalore

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zsj4oc%2BX7gAL3%2BZiFcmSmxCz46iKfPFpw%2By003DA1%2BxVw%40mail.gmail.com.

Re: [datameet] Help with R logic - near similar name

[Dilawar Singh]

Not sure what is the equivalent of python difflib (SequenceMatcher) in R. If 
you have one, it will work.

Sent from a handheld device. Pardon the brevity and typos.
On Aug 25, 2020, 20:09 +0530, rammano...@gmail.com , 
wrote:
> Hi,
>
> I have collected hospital data from multiple sources. However, each source 
> have different name. Trying to clean list with no duplicates. I am using R 
> and couldn't resolve with stringdist_join . Appreciate you suggesting some 
> approach.
>
> For example, Guntur (A.P) is listed with following names. Can we mark (or 
> eliminate) duplicate?
>
> Example 1
> SANKARA EYE HOSPITAL(GUNTUR)
> SANKARA EYE HOSPITAL
> SANKARA EYE HOSPITAL ( A UNIT OF SRI KANCHI KAMA KOTI MEDICAL TRUST)
>
>
> Example 2
> ASHIRWAD HEART HOSPITAL ( GHATKOPAR )
> Ashirwad Heart Hospital
> ASHIRWAD HEART HOSPITAL ( GHATKOPAR )
> Ashirwad Heart Hospita-Ghatkopar
>
> Thanks
> Ram
> --
> Datameet is a community of Data Science enthusiasts in India. Know more about 
> us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups 
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/datameet/19ee8101-84ec-42b0-974a-43035b5902f1n%40googlegroups.com.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/f69e252d-a5fb-4a34-afc3-67958614c8f3%40Spark.

[datameet] textricator | Generate structured data from PDFs

[Dilawar Singh]

Found this tool today. Can help you getting data from PDFs.

*Textricator* is a tool to extract text from documents and generate 
structured data.

If you have a bunch of PDFs with the same format (or one big, consistently 
formatted PDF) and you want to extract the data to CSV or JSON, 
*Textricator* can help! It can even work on OCR'ed documents!
https://github.com/measuresforjustice/textricator

best,
   Dilawar

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/cf8a3071-700f-4197-a86b-db698c910f80n%40googlegroups.com.

Re: [datameet] Get Smart Route to Avoid Containment Zone

[Dilawar Singh]

Thanks for the pointers Abhishek. The token can be obtained using a GET
request: "
https://covid19.quantela.com/qpa/1.0.0/public/token/bbmp.com/6a4d20c0-87dd-556b-9319-ab7147e388d9
";

I've created an automated pipeline for our institute App. Here is a
PHP+CURL solution to fetch the zone information:
https://github.com/dilawar/hippo/blob/master/system/extra/covid19.php

best,
Dilawar

On Sat, Jul 4, 2020 at 10:24 PM Abhishek  wrote:

> Hey Mohit,
> You can get the updated data from BBMP's dashboard
>
> curl --location --request POST '
> https://covid19.quantela.com/qpa/1.0.0/public/dashboard/getData/bbmp.com/7WGKgnEBHZrt-aeeDTHg/TmpE-XEBiNslxwBhF2EA
> ' \
> --header 'Authorization: Bearer 3a9d616a-5db2-3f0d-a2b6-5e2dd06a4037' \
> --header 'Content-Type: application/json' \
> --data-raw '{}'
>
> If this auth token expires you can obtain one from
> https://analysis.bbmpgov.in/#/dashboards/7WGKgnEBHZrt-aeeDTHg?embedd=1=0=bbmp.com
>
> Cheers
> -Abhishek
>
> On Sat, Jul 4, 2020 at 7:50 PM mohit K  wrote:
>
>> Hi Guys,
>>
>> Is there a place I can get Bangalore containment zones data?
>>
>> Appreciate your help.
>>
>> Mohit
>>
>> On Sun, May 17, 2020 at 8:35 PM Souvik Sankar Mitra <
>> souviksankar2...@gmail.com> wrote:
>>
>>> Thank you very much for your advice sir. I will try to implement of
>>> getting location by clicking. Actually sir I use Google's Direction API
>>> here. But I can try to implement it using open source routing.
>>> Thank you for the Stack Overflow link sir. It will help me..
>>>
>>> With regards ,
>>> Souvik Sankar Mitra
>>> M.Tech(Geoinformatics) , Indian Institute of Remote Sensing
>>>
>>> On Sunday, May 17, 2020 at 11:26:42 AM UTC+5:30, Nikhil VJ wrote:

 Hi Souvik,

 Good work!

 Suggestion on UI end: can you find a way to capture clicks or
 right-clicks on the map and let the user add to/from addresses that way,
 other than typing?
 That might also save on place api usage.

 I've made custom context menus / click-to-pin feature in Leaflet maps,
 not sure how its done in google maps but I've seen such implementations
 elsewhere.
 Here's a link from a quick search:
 https://stackoverflow.com/questions/7168394/google-map-v3-context-menu#10957262

 Another for-later suggestion: Is your routing running on google's api
 or other? If other, then it might be a good bet to change the ui to leaflet
 and avoid google's usage costs that hit when visitor count increases.

 --
 Cheers,
 Nikhil VJ
 https://nikhilvj.co.in


 On Wed, May 13, 2020 at 12:29 PM Souvik Sankar Mitra <
 souviksa...@gmail.com> wrote:

> I have developed a smart navigation application to avoid containment
> zone. It gives a safe route between source and destination which does not
> go through any containment zone. Till now it only available for Kolkata 
> and
> Howrah but it can be implemented for other cities also.
>
> You can check it here - http://shampa.covidrespons.in/
> For more information you can check out this video -
> https://www.youtube.com/watch?v=UtUaCDmJV_M
>
> Hope you like it.
> Thank you.
>
> Souvik Sankar Mitra
> M.Tech (Geoinformatics) , Indian Institute of Remote Sensing
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know
> more about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google
> Groups "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to data...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/41afc5f7-aafe-4451-9d94-1d09ab15feb3%40googlegroups.com
> 
> .
>
 --
>>> Datameet is a community of Data Science enthusiasts in India. Know more
>>> about us by visiting http://datameet.org
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "datameet" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to datameet+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/datameet/fb9b81da-98d0-4b05-9561-0084f6ce7a4f%40googlegroups.com
>>> 
>>> .
>>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to 

Re: [datameet] Bangalore zone wise data on COVID19 in machine readable format

[Dilawar Singh]

BBMP has its own portal 
https://bbmp.maps.arcgis.com/apps/opsdashboard/index.html#/4f4f20e852744b96b493528aab76777d

But I see no change in it for last few days. May be they are updating the 
location once in a while? Reaching out to them over email so far as been 
fruitless. 

If anyone knows more up to date database/portal/API, that would be greatly 
helpful. 

I have setup an alert system in my institute based on this data but it is 
only effective if coordinates of the new cases are in public domain and 
updated frequently.

best,
Dilawar

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/111f8512-0b1f-41f0-a43b-bb3faf23d482o%40googlegroups.com.

Re: [datameet] Bangalore zone wise data on COVID19 in machine readable format

[Dilawar Singh]

Thanks Souvik. 
Great. I was looking for the coordinates. But seems like they have private info 
of patients as well!

Thanks again. This portal will do.

Best,
    Dilawat

⁣Get BlueMail for Android ​

On 19 Jun 2020, 3:02 AM, at 3:02 AM, Souvik Sankar Mitra 
 wrote:
>I scraped  https://kgis.ksrsac.in/covid/ and got list of all patients
>in
>Karnataka.
>
>On Thu, Jun 18, 2020 at 5:35 PM Dilawar Singh
>
>wrote:
>
>> Hi DataMeet,
>>
>> On page 8 of this document
>> http://covid.bbmpgov.in/reports/report20200617.pdf there is a map
>which
>> shows location of active cases in Bangalore. Is this data available
>in
>> machine readable format?
>>
>> --
>> Dilawar
>> NCBS Bangalore
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know
>more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google
>Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it,
>send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>>
>https://groups.google.com/d/msgid/datameet/CAM72-Zv8paf6PY0Lge6pT-HtUsMnAFwtMwSkrs19QVh7bedVYA%40mail.gmail.com
>>
><https://groups.google.com/d/msgid/datameet/CAM72-Zv8paf6PY0Lge6pT-HtUsMnAFwtMwSkrs19QVh7bedVYA%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
>
>
>-- 
>With Regards ,
>Souvik Sankar Mitra,
>*M.Tech Remote Sensing & GIS (Specialisation in Geoinformatics),*
>*Indian Institute of Remote Sensing , Indian Space Research
>Organisation
>(ISRO),*
>*Dehradun - 248001*
>
>-- 
>Datameet is a community of Data Science enthusiasts in India. Know more
>about us by visiting http://datameet.org
>--- 
>You received this message because you are subscribed to the Google
>Groups "datameet" group.
>To unsubscribe from this group and stop receiving emails from it, send
>an email to datameet+unsubscr...@googlegroups.com.
>To view this discussion on the web visit
>https://groups.google.com/d/msgid/datameet/CAM88csWLQuiLHCm%3Drj8HyBRTZ8%2BEO0Kg%2BnXO2-Byi3XNqwrJRw%40mail.gmail.com.

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/8f3a27c8-f80d-4d4e-a9d9-2111fe8fb806%40gmail.com.

[datameet] Bangalore zone wise data on COVID19 in machine readable format

[Dilawar Singh]

Hi DataMeet,

On page 8 of this document
http://covid.bbmpgov.in/reports/report20200617.pdf there is a map which
shows location of active cases in Bangalore. Is this data available in
machine readable format?

-- 
Dilawar
NCBS Bangalore

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zv8paf6PY0Lge6pT-HtUsMnAFwtMwSkrs19QVh7bedVYA%40mail.gmail.com.

Re: [datameet] Re: Water level data extraction

[Dilawar Singh]

Web-scraping requires knowledge of some programming language. In essence,
it is an art of transforming HTML to a format you like. For common tasks,
you can find some tools easily on the net e.g., extract all tables from
this URL and save them as csv.

For example, you can get the html from the above link using `curl` and pass
it to `pandoc` (or html2text) to convert to plain text.

[dilawars@localhost Downloads]$ curl
https://vmc.gov.in/WaterLevel.aspx | pandoc -f html -t plain
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
100 14330  100 143300 0  33325  0 --:--:-- --:--:-- --:--:-- 33325
Water Level Statistics

Ajwa

Vishwamitri

  Date  TimeLevel (Feet)
   --- --
   17/06/2020   06:00  207.65
   17/06/2020   03:00  207.55
   17/06/2020   00:00  207.55

  Date  TimeLevel (Feet)
   --- --
   17/06/2020   06:00   4.00
   16/06/2020   21:00   4.00
   16/06/2020   18:00   4.00


Copy paste this in a Libreoffice-Calc or MS OFFICE spreadsheet and see if
it gets converted to a spreadsheet.

Anyway, this webpage does accept a POST request but only in encrypted
format. So it is essentially useless. Contact the author of
https://craigdsouza.github.io/data/IMD-Hourly-Precipitation-Data, he
scraped rainfall data from IMD website (2018). This data is probably of no
use to you but he might know a better source to get this data. I couldn't
locate time series data for a city on IMD website.

best,
   Dilawar



On Tue, Jun 16, 2020 at 10:42 PM Vikas Rana  wrote:

> I don't know how to scrape the data... Any guidance would be helpful.
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/a5f65448-059b-4132-828c-ff36ccc60310o%40googlegroups.com
> .
>


-- 
Dilawar
NCBS Bangalore

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zv6EybsrbjvLq9-%2BpuQK3L-Ypf0dD92JZGEBGBD0Xdr9Q%40mail.gmail.com.

[datameet] Re: Officially Approved India Map Borders?

[Dilawar Singh]

http://www.surveyofindia.gov.in/pages/display/122-outline-maps-of-india

Example PDF (it is a scan copy): 
http://www.surveyofindia.gov.in/files/36m_india.pdf . Most maps on this 
site are not machine readable. 

best,
   Dilawar 



On Tuesday, June 16, 2020 at 10:49:46 PM UTC+5:30, Curran Kelleher wrote:
>
> Greetings,
>
> I'm working on some mapping projects where the goal is to be able to 
> publish the map legally within India. To do this, we need to use the "Gov't 
> of India - Approved" borders for India (particularly disputed borders). The 
> closest we've come is to use GADM India Border 
> , but this is not officially approved by 
> Gov't of India, and makes some comprimises around disputed borders. We have 
> also used Natural Earth Disputed Borders 
> 
>  
> as an overlay, but this is also not the ideal solution.
>
> So, does anyone know how to find an officially approved map of India's 
> borders? Ideally machine readable (Shapefile, GeoJSON), but alternatively 
> as an image? A PDF? Even can you get any printed physical maps that are 
> officially approved? I have not been able to find these maps anywhere. Many 
> thanks in advance.  
> 
>
> Best regards,
> Curran
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/b8f797b8-0ab0-4bb4-9466-0389b51b4b0eo%40googlegroups.com.

Re: [datameet] Re: Open Data in India: In a Restrictive Copyright Regime, Voluntary Organisations Pitch in to Make Data Accessible

[Dilawar Singh]

Hi Thej, Datameet

After reading this, I went to SOI site and tried downloading few maps. I
could only find PDFs. And data.gov.in also love PDFs.

I wrote a couple of scripts which I used to figure out boundaries  and
extract data from curves in old PDF paper. It's a tricky business but I had
decent success (One python3 app is here
https://github.com/dilawar/PlotDigitizer). These scripts can also be
tweaked to fetch boundaries from PDF maps (no deep-learning business yet).
If datameet still struggling to find the boundaries from such maps, please
send a few samples to me. I can probably setup a web-service to upload the
map and get the GeoJSON if results are OK.

best,
Dilawar

On Tue, Jun 16, 2020 at 8:49 AM Naveen Francis  wrote:

> Nice writeup Thej,
>
> SOI point of view is completely different
> https://youtu.be/o1qMZZOia3c?t=3174
>
> 'Don't take data, but starting using SOI services.'
>
>
> On Monday, 15 June 2020 11:11:43 UTC+5:30, Thejesh GN wrote:
>>
>> Team,
>>
>> I wrote a piece for EPW about the status of Open Data and How communities
>> like ours. Thank you for being an awesome community. The article is open to
>> all.
>>
>> Open Data in India: In a Restrictive Copyright Regime, Voluntary
>> Organisations Pitch in to Make Data Accessible
>>
>> 
>>
>> https://www.epw.in/engage/article/voluntary-organisations-india-counteract-states-copyright-regime-open-data
>> 
>>
>> - This was pre covid-19, so a lot of things related to that are missing.
>> - Comments are welcome here on this thread or  on this blog post
>> 
>>
>>
>> Thej
>> --
>> Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
>> http://thejeshgn.com
>> GPG ID :  0xBFFC8DD3C06DD6B0
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/a5569aa9-6842-4a90-9127-33c9f603af50o%40googlegroups.com
> 
> .
>


-- 
Dilawar
NCBS Bangalore

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-ZuRcATxrShC8zBtGVofQ4Ku37o0E5qbVBAZUdZtXLV2%2Bg%40mail.gmail.com.

Re: [datameet] PoK elevation data | India's center of mass

[Dilawar Singh]

Dear Arun

Thanks. [1] is exactly what I need. I didn't even know what to search for.

The code was unnecessary convolutied to post directly. It compares various
sampling methods; I'll post a simplified version along with the results
soon.

Thanks again,
   Dilawar

On Fri, Jun 12, 2020 at 3:02 AM Arun Ganesh  wrote:

> If you share your code it would be easier to understand what exactly you
> need.
>
> My approach for this exercise would be to grab the SRTM dem data[1] and
> clip it to the India borders[2] using GDAL/QGIS.
>
> What you then have is a gridded raster with each pixel representing the
> elevation at that point. It seems like that is all the data you would need
> for your computation right?
>
> [1] http://srtm.csi.cgiar.org/srtmdata/
> [2] https://github.com/datameet/maps/tree/master/Country
>
> On Thu, Jun 11, 2020 at 3:33 PM Dilawar Singh 
> wrote:
>
>> Hi datameet,
>>
>> I was browsing through my old repositories. I once computed the center of
>> mass of the country using available elevation data (data is from here
>> http://www.geonames.org/data-sources.html, this data does not have PoK).
>> The image is attached. The center of mass lies very near to Jhansi without
>> PoK. It is likely to shift to the north after adding PoK since the
>> Himalayas are heavy!
>>
>> I am keen on finishing this exercise with PoK elevation data. Do we have
>> the elevation data of India in datameet repositories? If not, I can prepare
>> such data by appending data from PK elevation data and send it via a PR or
>> hand over the repository itself.
>>
>> cheers,
>> --
>> Dilawar
>> NCBS Bangalore
>>
>> --
>> Datameet is a community of Data Science enthusiasts in India. Know more
>> about us by visiting http://datameet.org
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/datameet/CAM72-ZvCeZSC5yAZKxvjjpZcGT%3DuYWPFMdyMasoH8uN2Mks-jg%40mail.gmail.com
>> <https://groups.google.com/d/msgid/datameet/CAM72-ZvCeZSC5yAZKxvjjpZcGT%3DuYWPFMdyMasoH8uN2Mks-jg%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/CA%2BGKQr2u_ryFFaks7efEZ8yYKALnFTMYEa8qGaP_J4yQY8%2BpyQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/datameet/CA%2BGKQr2u_ryFFaks7efEZ8yYKALnFTMYEa8qGaP_J4yQY8%2BpyQ%40mail.gmail.com?utm_medium=email_source=footer>
> .
>


-- 
Dilawar
NCBS Bangalore

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CAM72-Zu5XwsZEBPu%2BjopDg9L7R13i_rbyBLuy1dv5mQ9HL9J1w%40mail.gmail.com.

Re: [datameet] Re: BhuNaksha: source of cadastral maps

[Dilawar Singh]

The updated script is available publicly on Gitlab 
https://gitlab.com/snippets/1982028 . It has to be tweaked for different 
states. 

best,
   Dilawar 


On Monday, April 27, 2020 at 12:16:25 AM UTC+5:30, Dilawar Singh wrote:
>
> Hello Sharad
>
> Here you go: https://gitlab.com/dilawar/bhunaksha/-/tree/master 
>
> [image: canvas.png]
>
> [image: canvas.png]
> Currently only district images of MP are downloaded and added to 
> repository above. You can browse them (link above) or download as a zip 
> file from here: 
> https://gitlab.com/dilawar/bhunaksha/-/archive/v2020.04.26/bhunaksha-v2020.04.26.zip
>  
> .
>
> Couple of samples are attached with this email. I am not sure what exactly 
> is needed. If you can send couple of screenshots of what exactly is needed, 
> I can tweak this script a bit over next weekend.
>
> Note that canvas size is small since I was using a laptop. 
>
> PS: The script is released under GNU GPLv3.  
>
> best,
>Dilawar
>
> On Sunday, April 26, 2020 at 6:37:49 PM UTC+5:30, Sharad Lele (शरच्चंद्र 
> लेले) wrote:
>>
>> Dear Dilawar:
>>
>> Many thanks for sharing this. Unfortunately, I am a complete ignoramus 
>> when it comes to Python, selenium or even accessing stuff from GitHub :-[
>>
>> Is there any way that you (or anyone else) can help? (Sorry, probably 
>> asking too much here!)
>>
>> The site is http://bhunaksha.cg.nic.in/
>>
>> (It appears to have gone down today, should be up tomorrow or so).
>> Sharad
>>
>> On 26-04-2020 13:48, Dilawar Singh wrote:
>>
>> Its quite possible. I've used python+selenium to automate browser for a 
>> horribly designed website. You can tweak it for your usage: 
>> https://github.com/dilawar/GreenWatchData 
>>
>> best,
>> Dilawar 
>>
>>
>>
>> On Saturday, April 25, 2020 at 1:26:28 PM UTC+5:30, Sharad Lele wrote: 
>>>
>>> Dear All, 
>>> A followup question addressed to everyone on the DATAMEET group:
>>>
>>> Can someone write code to help download a large number of cadastral maps 
>>> from a Bhunaksha website (in our case Chhattisgarh)? As you know, the 
>>> website /portal is awkwardly organized: not only are the maps in png format 
>>> (which is okay) but they don't have individual file names, and one has to 
>>> access each village separately by choosing it from a list within a taluka 
>>> (and within that some Revenue Inspector category).
>>>
>>> Can someone write code to download and tag a bunch of maps 
>>> automatically? Say for one whole taluka?
>>>
>>> Sharad
>>>
>>> On Friday, 13 December 2019 19:09:02 UTC+5:30, Sharad Lele wrote: 
>>>>
>>>> Dear All,
>>>> I have discovered a source of cadastral maps for several states. This 
>>>> is mostly under the BhuNaksha initiative of NIC. They seem to have created 
>>>> separate websites for each state. Don't know for how many. I have found:
>>>>
>>>> Maharashtra: 
>>>> https://mahabhunakasha.mahabhumi.gov.in/bhunaksha/27/index.jsp
>>>> Madhya Pradesh: http://www.mpbhuabhilekh.nic.in/
>>>> Chhattisgarh: https://bhunaksha.cg.nic.in/#home-pane  (DOES NOT WORK 
>>>> RIGHT NOW)
>>>> Odisha: http://bhunakshaodisha.nic.in/bhunaksha/
>>>>
>>>> The BhuNaksha websites have a common format: you can choose district, 
>>>> taluka and village, and then the village cadastral map shows up (its in 
>>>> png 
>>>> format). You can click on a plot/parcel, and it will show the details of 
>>>> that plot: area, owner name, etc. You can generate  pdf report for that 
>>>> parcel. 
>>>>
>>>> Maharashtra version had added ability to show you the cadastral map in 
>>>> geo-rectified mode on top of a google-earth-like satellite image--quite 
>>>> useful.
>>>>
>>>> CATCH: a) the names of district, taluka, village etc are IN THE STATE 
>>>> SCRIPT: so devanagari for MP & Maha & CG, but Odiya for Odisha. So if you 
>>>> don't read that script--you are sunk
>>>> b) in most states (except Maharashtra!), it is not enough to know the 
>>>> district, taluka and village name. Within a taluka, you have know Revenue 
>>>> Inspector and then something called 'halka', and then choose from the 
>>>> villages within that halka. If you don't know this (RI and halka), you 
>>>> have 
>>>> to do brute force: search in each halka in each R

Re: [datameet] Re: BhuNaksha: source of cadastral maps

[Dilawar Singh]

Hello Sharad

Here you go: https://gitlab.com/dilawar/bhunaksha/-/tree/master 

[image: canvas.png]

[image: canvas.png]
Currently only district images of MP are downloaded and added to repository 
above. You can browse them (link above) or download as a zip file from 
here: 
https://gitlab.com/dilawar/bhunaksha/-/archive/v2020.04.26/bhunaksha-v2020.04.26.zip
 
.

Couple of samples are attached with this email. I am not sure what exactly 
is needed. If you can send couple of screenshots of what exactly is needed, 
I can tweak this script a bit over next weekend.

Note that canvas size is small since I was using a laptop. 

PS: The script is released under GNU GPLv3.  

best,
   Dilawar

On Sunday, April 26, 2020 at 6:37:49 PM UTC+5:30, Sharad Lele (शरच्चंद्र 
लेले) wrote:
>
> Dear Dilawar:
>
> Many thanks for sharing this. Unfortunately, I am a complete ignoramus 
> when it comes to Python, selenium or even accessing stuff from GitHub :-[
>
> Is there any way that you (or anyone else) can help? (Sorry, probably 
> asking too much here!)
>
> The site is http://bhunaksha.cg.nic.in/
>
> (It appears to have gone down today, should be up tomorrow or so).
> Sharad
>
> On 26-04-2020 13:48, Dilawar Singh wrote:
>
> Its quite possible. I've used python+selenium to automate browser for a 
> horribly designed website. You can tweak it for your usage: 
> https://github.com/dilawar/GreenWatchData 
>
> best,
> Dilawar 
>
>
>
> On Saturday, April 25, 2020 at 1:26:28 PM UTC+5:30, Sharad Lele wrote: 
>>
>> Dear All, 
>> A followup question addressed to everyone on the DATAMEET group:
>>
>> Can someone write code to help download a large number of cadastral maps 
>> from a Bhunaksha website (in our case Chhattisgarh)? As you know, the 
>> website /portal is awkwardly organized: not only are the maps in png format 
>> (which is okay) but they don't have individual file names, and one has to 
>> access each village separately by choosing it from a list within a taluka 
>> (and within that some Revenue Inspector category).
>>
>> Can someone write code to download and tag a bunch of maps automatically? 
>> Say for one whole taluka?
>>
>> Sharad
>>
>> On Friday, 13 December 2019 19:09:02 UTC+5:30, Sharad Lele wrote: 
>>>
>>> Dear All,
>>> I have discovered a source of cadastral maps for several states. This is 
>>> mostly under the BhuNaksha initiative of NIC. They seem to have created 
>>> separate websites for each state. Don't know for how many. I have found:
>>>
>>> Maharashtra: 
>>> https://mahabhunakasha.mahabhumi.gov.in/bhunaksha/27/index.jsp
>>> Madhya Pradesh: http://www.mpbhuabhilekh.nic.in/
>>> Chhattisgarh: https://bhunaksha.cg.nic.in/#home-pane  (DOES NOT WORK 
>>> RIGHT NOW)
>>> Odisha: http://bhunakshaodisha.nic.in/bhunaksha/
>>>
>>> The BhuNaksha websites have a common format: you can choose district, 
>>> taluka and village, and then the village cadastral map shows up (its in png 
>>> format). You can click on a plot/parcel, and it will show the details of 
>>> that plot: area, owner name, etc. You can generate  pdf report for that 
>>> parcel. 
>>>
>>> Maharashtra version had added ability to show you the cadastral map in 
>>> geo-rectified mode on top of a google-earth-like satellite image--quite 
>>> useful.
>>>
>>> CATCH: a) the names of district, taluka, village etc are IN THE STATE 
>>> SCRIPT: so devanagari for MP & Maha & CG, but Odiya for Odisha. So if you 
>>> don't read that script--you are sunk
>>> b) in most states (except Maharashtra!), it is not enough to know the 
>>> district, taluka and village name. Within a taluka, you have know Revenue 
>>> Inspector and then something called 'halka', and then choose from the 
>>> villages within that halka. If you don't know this (RI and halka), you have 
>>> to do brute force: search in each halka in each RI! (typical sarkari 
>>> rubbish: not wanting common people to actually use the website!).
>>>
>>> Karnataka seems to have its own initiative: 
>>> https://www.landrecords.karnataka.gov.in/service3/
>>> Here, you can actually download pdfs of each cadastral map, and in some 
>>> cases, even the KML! And searching for the village is district, taluka and 
>>> then village.
>>>
>>> Hope this helps.
>>>
>>> I am sure there are more such websites for other states. Please look up 
>>> and share.
>>>
>>> Sharad
>>>
>> -- 
> Datameet is a community o

[datameet] Re: BhuNaksha: source of cadastral maps

[Dilawar Singh]

Its quite possible. I've used python+selenium to automate browser for a 
horribly designed website. You can tweak it for your usage: 
https://github.com/dilawar/GreenWatchData 

best,
Dilawar 



On Saturday, April 25, 2020 at 1:26:28 PM UTC+5:30, Sharad Lele wrote:
>
> Dear All,
> A followup question addressed to everyone on the DATAMEET group:
>
> Can someone write code to help download a large number of cadastral maps 
> from a Bhunaksha website (in our case Chhattisgarh)? As you know, the 
> website /portal is awkwardly organized: not only are the maps in png format 
> (which is okay) but they don't have individual file names, and one has to 
> access each village separately by choosing it from a list within a taluka 
> (and within that some Revenue Inspector category).
>
> Can someone write code to download and tag a bunch of maps automatically? 
> Say for one whole taluka?
>
> Sharad
>
> On Friday, 13 December 2019 19:09:02 UTC+5:30, Sharad Lele wrote:
>>
>> Dear All,
>> I have discovered a source of cadastral maps for several states. This is 
>> mostly under the BhuNaksha initiative of NIC. They seem to have created 
>> separate websites for each state. Don't know for how many. I have found:
>>
>> Maharashtra: 
>> https://mahabhunakasha.mahabhumi.gov.in/bhunaksha/27/index.jsp
>> Madhya Pradesh: http://www.mpbhuabhilekh.nic.in/
>> Chhattisgarh: https://bhunaksha.cg.nic.in/#home-pane  (DOES NOT WORK 
>> RIGHT NOW)
>> Odisha: http://bhunakshaodisha.nic.in/bhunaksha/
>>
>> The BhuNaksha websites have a common format: you can choose district, 
>> taluka and village, and then the village cadastral map shows up (its in png 
>> format). You can click on a plot/parcel, and it will show the details of 
>> that plot: area, owner name, etc. You can generate  pdf report for that 
>> parcel. 
>>
>> Maharashtra version had added ability to show you the cadastral map in 
>> geo-rectified mode on top of a google-earth-like satellite image--quite 
>> useful.
>>
>> CATCH: a) the names of district, taluka, village etc are IN THE STATE 
>> SCRIPT: so devanagari for MP & Maha & CG, but Odiya for Odisha. So if you 
>> don't read that script--you are sunk
>> b) in most states (except Maharashtra!), it is not enough to know the 
>> district, taluka and village name. Within a taluka, you have know Revenue 
>> Inspector and then something called 'halka', and then choose from the 
>> villages within that halka. If you don't know this (RI and halka), you have 
>> to do brute force: search in each halka in each RI! (typical sarkari 
>> rubbish: not wanting common people to actually use the website!).
>>
>> Karnataka seems to have its own initiative: 
>> https://www.landrecords.karnataka.gov.in/service3/
>> Here, you can actually download pdfs of each cadastral map, and in some 
>> cases, even the KML! And searching for the village is district, taluka and 
>> then village.
>>
>> Hope this helps.
>>
>> I am sure there are more such websites for other states. Please look up 
>> and share.
>>
>> Sharad
>>
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/8ba4688b-25fd-4d01-a4da-5adc31da1459%40googlegroups.com.