Re: [datameet] Re: 2001 village and town boundaries

[Digvijay Bendrikar Shinde]

Connecting Prof Jitendra Shah from GISE Lab of CSE IIT Bombay. He may like
to discuss this with you.

About RIDDHI foundation, it's an organisation based in Kolkata. They work
in GIS technologies to support social projects.

Regards
Digvijay



On Mon, Jun 1, 2020, 9:42 AM Sharad Lele (शरच्चंद्र लेले) <
sharad.l...@gmail.com> wrote:

> Dear Digvijay:
>
> Thanks for this information. I did not know about the censusgis.org
> portal. It seems to have boundaries for areas where I have otherwise drawn
> a blank. BUT it is really hard to use this portal for anything other than
> to 'see' approximately what is going on. Pity. Do you know who RIDDHI is?
> It appears to have been authorised by Census of India to put this up.
>
> You may also notice that the IITB maps are *different* from the Census
> maps. They appear to come from the MRSAC maps that are present on the MRSAC
> portal. (Not sure why CSE-IITB folks have not indicated the source). These
> maps/boundaries are in my opinion much more accurate (both in terms of
> their shape and their geo-positioning) as compared to Census maps, but that
> leaves certain questions about missing villages unanswered... The missing
> polygons in these maps appear to be either forest polygons or town
> polygons, and am curious why they have gone missing. Is there someone from
> CSE-IITB team that we can loop in on this?
>
> Sharad
>
> On 31-05-2020 13:37, Digvijay Bendrikar Shinde wrote:
>
> Hello Prof Mark,
>
> Thank you for the resource.
>
> Have you seen the India GIS portal? http://www.censusgis.org/india/ it
> has the census data of 2001 and 2011 integrated with (up to) Village level
> shapefiles. you can make basic spatial viz using this. But files can not be
> downloaded.
>
> Also, CSE department, IIT Bombay has put Maharashtra state's Census '11
> data integrated village level shapefiles here
> https://www.cse.iitb.ac.in/~pocra/MahaCensus_shapefile_data1.2/Boundary.html
>
> Hope this helps in you.
>
> Regards,
> Digvijay
> PhD Scholar
> CTARA, IIT Bombay
>
> On Sun, May 31, 2020 at 1:07 PM Sharad Lele  wrote:
>
>> Dear Mark,
>> Happy to know about your interest. I am also quite interested in these
>> issues, having worked on 1991, 2001 and 2011 census datasets and their
>> spatial representation (at least for Karnataka and some other states).
>> There are many issues, both with the census datasets themselves and with
>> the spatial boundary datasets released by Meiyyappan et al. I may not be
>> able to lay out everything immediately, because of being in the throes of
>> some deadlines, but hope to go through your writeup and respond a bit
>> later--maybe mid-June, if that is okay with you.
>>
>> Best,
>> Sharad
>>
>> On Sunday, May 31, 2020 at 1:09:24 AM UTC+5:30, Mark Montgomery wrote:
>>>
>>> Let me introduce myself to the group in this way: I am an Economics
>>> professor at Stony Brook University in New York, with a long-time interest
>>> in Indian urbanization. I am also keen to see as much as possible of the
>>> spatial and socioeconomic detail on urbanization placed in the public
>>> domain. Toward that end, colleagues and I have been knitting together the
>>> 2001 and 2011 primary census abstracts (PCAs) that the Indian census
>>> authorities have made available on the census website and incorporating
>>> published data from the District Census Handbooks, all of these at the
>>> level of individual settlements with coverage of wards for the PCAs. Our
>>> aim is to create an integrated and publicly-accessible database based only
>>> on publicly-available sources. As you would know very well, the spatial
>>> side of the task is more challenging for 2001 than 2011.
>>>
>>> At the moment, I seek your guidance on the remarkable DataMeet
>>> collection of polygons for villages, census towns, and statutory urban
>>> centers, to which a number of you have contributed months or even years of
>>> effort. I have linked your spatial records to the PCA identifiers
>>> (including subdistrict and district) and in the process have come across
>>> some issues (mainly concerning the vintages of the maps that were used, and
>>> various oddities regarding identifiers) that some of you may know about. My
>>> own spatial work uses R, but I am happy to share these results with the
>>> group in other spatial formats (for instance, as geojson or geopackage
>>> files). The next steps I have in mind are to compare the DataMeet polygons
>>> with the often-mentioned Meiyappan et al. (2018) polygons that have been
>>> publicly available at the Socioeconomic Data Applications Center (SEDAC)
>>> site since 2018, and with a lesser-known but evidently high-quality
>>> collection of 2001 point coordinates for villages and some hamlets
>>> assembled by a University of Tokyo history professor and available on his
>>> website.
>>>
>>> I'm attaching a short pdf that explains these three public-domain
>>> sources (with links to the SEDAC and Univ. of Tokyo sources, and with a
>>> critical review 

Re: [datameet] Re: 2001 village and town boundaries

[शरच्चंद्र लेले]

Dear Digvijay:

Thanks for this information. I did not know about the censusgis.org 
portal. It seems to have boundaries for areas where I have otherwise 
drawn a blank. BUT it is really hard to use this portal for anything 
other than to 'see' approximately what is going on. Pity. Do you know 
who RIDDHI is? It appears to have been authorised by Census of India to 
put this up.


You may also notice that the IITB maps are _different_ from the Census 
maps. They appear to come from the MRSAC maps that are present on the 
MRSAC portal. (Not sure why CSE-IITB folks have not indicated the 
source). These maps/boundaries are in my opinion much more accurate 
(both in terms of their shape and their geo-positioning) as compared to 
Census maps, but that leaves certain questions about missing villages 
unanswered... The missing polygons in these maps appear to be either 
forest polygons or town polygons, and am curious why they have gone 
missing. Is there someone from CSE-IITB team that we can loop in on this?


Sharad


On 31-05-2020 13:37, Digvijay Bendrikar Shinde wrote:

Hello Prof Mark,

Thank you for the resource.

Have you seen the India GIS portal? http://www.censusgis.org/india/ it 
has the census data of 2001 and 2011 integrated with (up to) Village 
level shapefiles. you can make basic spatial viz using this. But files 
can not be downloaded.


Also, CSE department, IIT Bombay has put Maharashtra state's Census 
'11 data integrated village level shapefiles here 
https://www.cse.iitb.ac.in/~pocra/MahaCensus_shapefile_data1.2/Boundary.html 



Hope this helps in you.

Regards,
Digvijay
PhD Scholar
CTARA, IIT Bombay

On Sun, May 31, 2020 at 1:07 PM Sharad Lele > wrote:


Dear Mark,
Happy to know about your interest. I am also quite interested in
these issues, having worked on 1991, 2001 and 2011 census datasets
and their spatial representation (at least for Karnataka and some
other states). There are many issues, both with the census
datasets themselves and with the spatial boundary datasets
released by Meiyyappan et al. I may not be able to lay out
everything immediately, because of being in the throes of some
deadlines, but hope to go through your writeup and respond a bit
later--maybe mid-June, if that is okay with you.

Best,
Sharad

On Sunday, May 31, 2020 at 1:09:24 AM UTC+5:30, Mark Montgomery
wrote:

Let me introduce myself to the group in this way: I am an
Economics professor at Stony Brook University in New York,
with a long-time interest in Indian urbanization. I am also
keen to see as much as possible of the spatial and
socioeconomic detail on urbanization placed in the public
domain. Toward that end, colleagues and I have been knitting
together the 2001 and 2011 primary census abstracts (PCAs)
that the Indian census authorities have made available on the
census website and incorporating published data from the
District Census Handbooks, all of these at the level of
individual settlements with coverage of wards for the PCAs.
Our aim is to create an integrated and publicly-accessible
database based only on publicly-available sources. As you
would know very well, the spatial side of the task is more
challenging for 2001 than 2011.

At the moment, I seek your guidance on the remarkable DataMeet
collection of polygons for villages, census towns, and
statutory urban centers, to which a number of you have
contributed months or even years of effort. I have linked your
spatial records to the PCA identifiers (including subdistrict
and district) and in the process have come across some issues
(mainly concerning the vintages of the maps that were used,
and various oddities regarding identifiers) that some of you
may know about. My own spatial work uses R, but I am happy to
share these results with the group in other spatial formats
(for instance, as geojson or geopackage files). The next steps
I have in mind are to compare the DataMeet polygons with the
often-mentioned Meiyappan et al. (2018) polygons that have
been publicly available at the Socioeconomic Data Applications
Center (SEDAC) site since 2018, and with a lesser-known but
evidently high-quality collection of 2001 point coordinates
for villages and some hamlets assembled by a University of
Tokyo history professor and available on his website.

I'm attaching a short pdf that explains these three
public-domain sources (with links to the SEDAC and Univ. of
Tokyo sources, and with a critical review of aspects of those
spatial datasets), and which in particular lays out some of
the 

Re: [datameet] Re: 2001 village and town boundaries

[Mark Montgomery]

Many thanks to Digvijay and Sharad, and I hope to continue sharing ideas 
and data with you.  I'm excited to hear about the census gis site and the 
availability of integrated files for Maharashtra; I did not know about 
either resource.

One issue that I am concerned about is whether it will be possible to 
create boundary files for 2001 villages and (especially) statutory towns 
that respect subdistrict borders. The aim is to link the spatial boundaries 
as closely as possible with the PCAs, which provide the detailed 
settlement-within-subdistrict counts. I doubt that this can be done 
perfectly with the spatial data we have, but it might prove adequate to 
overlay the 2001 subdistrict boundaries (one version of which was supplied 
on Justin Elliot Meyers' Github, announced to this group some time back) on 
top of the settlement boundaries, so as to approximate the portions of a 
settlement that lie within each subdistrict. I hope to experiment with this 
idea later this week to see if it is over-ambitious.

Best regards, Mark

On Sunday, May 31, 2020 at 10:37:29 AM UTC-4, Digvijay Bendrikar Shinde 
wrote:
>
> Hello Prof Mark,
>
> Thank you for the resource.
>
> Have you seen the India GIS portal? http://www.censusgis.org/india/ it 
> has the census data of 2001 and 2011 integrated with (up to) Village level 
> shapefiles. you can make basic spatial viz using this. But files can not be 
> downloaded.
>
> Also, CSE department, IIT Bombay has put Maharashtra state's Census '11 
> data integrated village level shapefiles here 
> https://www.cse.iitb.ac.in/~pocra/MahaCensus_shapefile_data1.2/Boundary.html
>
> Hope this helps in you.
>
> Regards,
> Digvijay
> PhD Scholar
> CTARA, IIT Bombay
>
> On Sun, May 31, 2020 at 1:07 PM Sharad Lele  > wrote:
>
>> Dear Mark,
>> Happy to know about your interest. I am also quite interested in these 
>> issues, having worked on 1991, 2001 and 2011 census datasets and their 
>> spatial representation (at least for Karnataka and some other states). 
>> There are many issues, both with the census datasets themselves and with 
>> the spatial boundary datasets released by Meiyyappan et al. I may not be 
>> able to lay out everything immediately, because of being in the throes of 
>> some deadlines, but hope to go through your writeup and respond a bit 
>> later--maybe mid-June, if that is okay with you.
>>
>> Best,
>> Sharad
>>
>> On Sunday, May 31, 2020 at 1:09:24 AM UTC+5:30, Mark Montgomery wrote:
>>>
>>> Let me introduce myself to the group in this way: I am an Economics 
>>> professor at Stony Brook University in New York, with a long-time interest 
>>> in Indian urbanization. I am also keen to see as much as possible of the 
>>> spatial and socioeconomic detail on urbanization placed in the public 
>>> domain. Toward that end, colleagues and I have been knitting together the 
>>> 2001 and 2011 primary census abstracts (PCAs) that the Indian census 
>>> authorities have made available on the census website and incorporating 
>>> published data from the District Census Handbooks, all of these at the 
>>> level of individual settlements with coverage of wards for the PCAs. Our 
>>> aim is to create an integrated and publicly-accessible database based only 
>>> on publicly-available sources. As you would know very well, the spatial 
>>> side of the task is more challenging for 2001 than 2011.
>>>
>>> At the moment, I seek your guidance on the remarkable DataMeet 
>>> collection of polygons for villages, census towns, and statutory urban 
>>> centers, to which a number of you have contributed months or even years of 
>>> effort. I have linked your spatial records to the PCA identifiers 
>>> (including subdistrict and district) and in the process have come across 
>>> some issues (mainly concerning the vintages of the maps that were used, and 
>>> various oddities regarding identifiers) that some of you may know about. My 
>>> own spatial work uses R, but I am happy to share these results with the 
>>> group in other spatial formats (for instance, as geojson or geopackage 
>>> files). The next steps I have in mind are to compare the DataMeet polygons 
>>> with the often-mentioned Meiyappan et al. (2018) polygons that have been 
>>> publicly available at the Socioeconomic Data Applications Center (SEDAC) 
>>> site since 2018, and with a lesser-known but evidently high-quality 
>>> collection of 2001 point coordinates for villages and some hamlets 
>>> assembled by a University of Tokyo history professor and available on his 
>>> website.
>>>
>>> I'm attaching a short pdf that explains these three public-domain 
>>> sources (with links to the SEDAC and Univ. of Tokyo sources, and with a 
>>> critical review of aspects of those spatial datasets), and which in 
>>> particular lays out some of the issues I've encountered with the DataMeet 
>>> collection. (I've yet to get to grips with the Karnataka data for 1991, and 
>>> with the Rajasthan data that I believe are for 2011 or 

Re: [datameet] Re: 2001 village and town boundaries

[Digvijay Bendrikar Shinde]

Hello Prof Mark,

Thank you for the resource.

Have you seen the India GIS portal? http://www.censusgis.org/india/ it has
the census data of 2001 and 2011 integrated with (up to) Village level
shapefiles. you can make basic spatial viz using this. But files can not be
downloaded.

Also, CSE department, IIT Bombay has put Maharashtra state's Census '11
data integrated village level shapefiles here
https://www.cse.iitb.ac.in/~pocra/MahaCensus_shapefile_data1.2/Boundary.html

Hope this helps in you.

Regards,
Digvijay
PhD Scholar
CTARA, IIT Bombay

On Sun, May 31, 2020 at 1:07 PM Sharad Lele  wrote:

> Dear Mark,
> Happy to know about your interest. I am also quite interested in these
> issues, having worked on 1991, 2001 and 2011 census datasets and their
> spatial representation (at least for Karnataka and some other states).
> There are many issues, both with the census datasets themselves and with
> the spatial boundary datasets released by Meiyyappan et al. I may not be
> able to lay out everything immediately, because of being in the throes of
> some deadlines, but hope to go through your writeup and respond a bit
> later--maybe mid-June, if that is okay with you.
>
> Best,
> Sharad
>
> On Sunday, May 31, 2020 at 1:09:24 AM UTC+5:30, Mark Montgomery wrote:
>>
>> Let me introduce myself to the group in this way: I am an Economics
>> professor at Stony Brook University in New York, with a long-time interest
>> in Indian urbanization. I am also keen to see as much as possible of the
>> spatial and socioeconomic detail on urbanization placed in the public
>> domain. Toward that end, colleagues and I have been knitting together the
>> 2001 and 2011 primary census abstracts (PCAs) that the Indian census
>> authorities have made available on the census website and incorporating
>> published data from the District Census Handbooks, all of these at the
>> level of individual settlements with coverage of wards for the PCAs. Our
>> aim is to create an integrated and publicly-accessible database based only
>> on publicly-available sources. As you would know very well, the spatial
>> side of the task is more challenging for 2001 than 2011.
>>
>> At the moment, I seek your guidance on the remarkable DataMeet collection
>> of polygons for villages, census towns, and statutory urban centers, to
>> which a number of you have contributed months or even years of effort. I
>> have linked your spatial records to the PCA identifiers (including
>> subdistrict and district) and in the process have come across some issues
>> (mainly concerning the vintages of the maps that were used, and various
>> oddities regarding identifiers) that some of you may know about. My own
>> spatial work uses R, but I am happy to share these results with the group
>> in other spatial formats (for instance, as geojson or geopackage files).
>> The next steps I have in mind are to compare the DataMeet polygons with the
>> often-mentioned Meiyappan et al. (2018) polygons that have been publicly
>> available at the Socioeconomic Data Applications Center (SEDAC) site since
>> 2018, and with a lesser-known but evidently high-quality collection of 2001
>> point coordinates for villages and some hamlets assembled by a University
>> of Tokyo history professor and available on his website.
>>
>> I'm attaching a short pdf that explains these three public-domain sources
>> (with links to the SEDAC and Univ. of Tokyo sources, and with a critical
>> review of aspects of those spatial datasets), and which in particular lays
>> out some of the issues I've encountered with the DataMeet collection. (I've
>> yet to get to grips with the Karnataka data for 1991, and with the
>> Rajasthan data that I believe are for 2011 or later.) I would be really
>> grateful for criticism and suggestions!
>>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/datameet/515058c6-e520-405c-8611-e7cfc94a4e91%40googlegroups.com
> 
> .
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/CA%2BsGXOKhw1OmFwgRiZC5AAoWezaOTB1b70xLu_TZYgP%3DGdvkYg%40mail.gmail.com.

[datameet] Re: 2001 village and town boundaries

[Sharad Lele]

Dear Mark,
Happy to know about your interest. I am also quite interested in these 
issues, having worked on 1991, 2001 and 2011 census datasets and their 
spatial representation (at least for Karnataka and some other states). 
There are many issues, both with the census datasets themselves and with 
the spatial boundary datasets released by Meiyyappan et al. I may not be 
able to lay out everything immediately, because of being in the throes of 
some deadlines, but hope to go through your writeup and respond a bit 
later--maybe mid-June, if that is okay with you.

Best,
Sharad

On Sunday, May 31, 2020 at 1:09:24 AM UTC+5:30, Mark Montgomery wrote:
>
> Let me introduce myself to the group in this way: I am an Economics 
> professor at Stony Brook University in New York, with a long-time interest 
> in Indian urbanization. I am also keen to see as much as possible of the 
> spatial and socioeconomic detail on urbanization placed in the public 
> domain. Toward that end, colleagues and I have been knitting together the 
> 2001 and 2011 primary census abstracts (PCAs) that the Indian census 
> authorities have made available on the census website and incorporating 
> published data from the District Census Handbooks, all of these at the 
> level of individual settlements with coverage of wards for the PCAs. Our 
> aim is to create an integrated and publicly-accessible database based only 
> on publicly-available sources. As you would know very well, the spatial 
> side of the task is more challenging for 2001 than 2011.
>
> At the moment, I seek your guidance on the remarkable DataMeet collection 
> of polygons for villages, census towns, and statutory urban centers, to 
> which a number of you have contributed months or even years of effort. I 
> have linked your spatial records to the PCA identifiers (including 
> subdistrict and district) and in the process have come across some issues 
> (mainly concerning the vintages of the maps that were used, and various 
> oddities regarding identifiers) that some of you may know about. My own 
> spatial work uses R, but I am happy to share these results with the group 
> in other spatial formats (for instance, as geojson or geopackage files). 
> The next steps I have in mind are to compare the DataMeet polygons with the 
> often-mentioned Meiyappan et al. (2018) polygons that have been publicly 
> available at the Socioeconomic Data Applications Center (SEDAC) site since 
> 2018, and with a lesser-known but evidently high-quality collection of 2001 
> point coordinates for villages and some hamlets assembled by a University 
> of Tokyo history professor and available on his website.
>
> I'm attaching a short pdf that explains these three public-domain sources 
> (with links to the SEDAC and Univ. of Tokyo sources, and with a critical 
> review of aspects of those spatial datasets), and which in particular lays 
> out some of the issues I've encountered with the DataMeet collection. (I've 
> yet to get to grips with the Karnataka data for 1991, and with the 
> Rajasthan data that I believe are for 2011 or later.) I would be really 
> grateful for criticism and suggestions!
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/datameet/515058c6-e520-405c-8611-e7cfc94a4e91%40googlegroups.com.