Re: [datameet] Invitation | ORF Event: Better Data for Better Health | Monday, July 25

2016-07-18 Thread Gora Mohanty 
Hi,

What is the relevance to open data? Which of these data will be openly
accessible?

Regards,
Gora


On 18 July 2016 at 14:10, oommen  wrote:

>
> 
>
> Observer Research Foundation is pleased to invite you for the event, *Better
> Data for Better Health — Developing an Indian Approach on Indicators to
> Achieve Sustainable Development Goals*, to be held on *July 25, **10.30
> am to 1.30 pm* at *Longchamp*, *Taj Mahal Hotel*, *New Delhi*.
>
> A major challenge to health policy-making in India is the insufficient
> district level data for customised and evidence-based planning.  Better
> data and statistics help track progress, strengthen accountability and
> support evidence-based decisions. A data revolution is needed in order to
> draw on existing and new sources of data to fully integrate statistics into
> decision making — to promote open access to, and use of, data and ensure
> increased support for statistical systems.
>
> As part of the Foundation’s work on the Sustainable Development Goals,
> this event will kick off ORF’s *Health Forum* by engaging senior
> policymakers from the Central Government, representatives of the industry,
> civil society, international organisations, academia and media. A *panel
> discussion* at the event will deliberate upon how quality data and a
> robust national indicator framework can help India achieve its health and
> nutrition goals quicker.
>
> Please *find attached the agenda* for the event.
>
> We request you to confirm your participation to Priyanka Shah <
> priyanka.s...@orfonline.org> at the earliest.
>
> Looking forward to your participation.
> Warm Regards,
> Team ORF
>
> (Photo courtesy: DFID/Flickr/CC BY-SA 2.0
> )
> Research   |  Forums
>  | Events
> |  People
>   |  Expert Speak
>   |  About
> 
> 
> 
> 
> 
> *Copyright © 2016 Observer Research Foundation, All rights reserved.*
>
> *Our mailing address is:*
> 20, Rouse Avenue Institutional Area,
> New Delhi-110002, India
> Phone: +91 11 4352 0020
> Website: http://orfonline.org
>
> You can update your preferences or unsubscribe from this list
>
>
> Oommen C. Kurian
> Fellow, Public Health
>
> Observer Research Foundation
> 20 Rouse Avenue Institutional Area, New Delhi, 110002
> Off: 011-43520020; Fax: 011-43520003
>
> --
> Datameet is a community of Data Science enthusiasts in India. Know more
> about us by visiting http://datameet.org
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Public consultation on Government Open Data Use License - India

2016-06-27 Thread Gora Mohanty 
On 28 June 2016 at 10:19, Devdatta Tengshe  wrote:

> IMNAL, but Overall this license looks quite ok, except for 6.c & 6.e
>

6.c and 6.e specifically do not seem too worrisome to me, but the entirety
of point 6 is kind of strange: where are datasets that have personal
information, patent/trademark infringements, and that the provider is not
licensed to supply coming from? Is the GoI saying that should the datasets
on data.gov.in include such data, it is the user's job to exclude them.

Also, 3a seems a little vague to me: what are "non-exclusive processes"?
Suppose I have a website that uses such data, and requires users to log in
to see the data: is that exclusive?

All, in all, kudos to data.gov.in for taking a huge step forward in making
their data accessible.

Regards,
Gora

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Re: How to create horizontal bar visualizations

2015-05-07 Thread Gora Mohanty 
On 7 May 2015 at 13:12, Aadesh Sachan aadesh...@gmail.com wrote:
 Hi Navneethan,

 Thanks for replying.

 Here I am trying to create static visualization on a website/web app by
 taking some input numbers from user and creating a static visualization
 based on the numbers..

What technology does the website use?

The easiest way is probably a Javascript graphing library. Almost any
library should handle simple bar graphs, so you might want to go with
the simplest one. If you think that you might work with more complex
graphs later, I would suggest using d3, along with a library like
rickshaw that rides on top of d3.

Regards,
Gora

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] [Delhi] How about a meet-up?

2014-07-20 Thread Gora Mohanty 
On 20 July 2014 14:01, sumandro suman...@gmail.com wrote:

 Dear Delhi DataMeet-niks,

 Hope you all are well, and not too tired of our staple shapefile discussions.

 Am writing this email to suggest that we do a planning meeting this week or 
 early next week.

 Possible dates are Friday, July 25, or Tuesday, July 29.

 What can be a possible venue?

 Sarai (#29 Rajpur Road, Civil Lines Metro Station) is always available. But 
 maybe somewhere more central is better.

 Can the Akvo people host us on either days? Isha/Amitangshu, can you please 
 confirm? Surendran’s workplace is also an option.
[...]

ILUG-Delhi, and the NCR Python User's Group are having a joint meeting
on Sat., the 26th, at JNU, from 2-5pm. Should we consider also joining
that: Might help in cross-fertilisation of ideas, or might just be too
many diverging interests. What are people's thoughts?

Regards,
Gora

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Map + Data + Video

2014-06-04 Thread Gora Mohanty 
On 4 June 2014 22:51, Ankur Nagar ankur.na...@gmail.com wrote:

 Hi,

 A quick question - would you know anyone in India (elsewhere is ok too, India 
 is preferable) who would be able to develop a short video that uses maps + 
 open data to highlight the earthquake risk in and around India.  Something 
 like this:

 https://www.youtube.com/watch?v=NSBjEvPH2j4

 Do drop me a line if you know someone.

This looks interesting, and we might want to get involved, but could
you expand further on this?
* Are the earthquake-related data available?
* What are you looking for, i.e., who would be using the presented
results? Why a video specifically, as opposed to, say an online GIS
interface?

Regards,
Gora

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Map + Data + Video

2014-06-04 Thread Gora Mohanty 
On 4 June 2014 23:48, Ankur Nagar ankur.na...@gmail.com wrote:
 Hi Nisha - yes, we the data is available.

Great. Sounds interesting. What format are the data in? How does one
get geographical information for the data?

 Hi Gora, not sure if we have been introduced - there are a number of
 possibilities one can imagine here from a GIS interlace to JS-based
 visualization to an app and so on. Our objective for a video is to drive the
 message home in a quick, impactful way to policy audiences who may not be
 tech savvy and would be time strapped.

* The only introduction one should need is if there are mutual points
of interest :-)
* Our primary interest would be in an online interface
* A video can be (manually) generated from such an interface. As you
mention, there is a valid reason why one might want a video. But, I
will ask again my earlier question: Who do you think your primary
audience would be?

Regards,
Gora

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] [Article] Limitations of the PDF

2014-06-01 Thread Gora Mohanty 
On 1 June 2014 11:16, Venkata Pingali ping...@gmail.com wrote:

 As it turns out, I am working on a DSL for data extraction for
 a client - pretty much what you said with some nuances. The
 client is open-source friendly and I will request for open
 sourcing the tooling.


The PDF specification is huge, and parsing a PDF is no simple task.
We use Python, and have had decent luck with pdfminer for parsing
text PDFs. The documentation is a little sketchy, but one can find blogs
that describe how pdfminer can be used.

Regards,
Gora

-- 
Datameet is a community of Data Science enthusiasts in India. Know more about 
us by visiting http://datameet.org
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [datameet] Tree survey in bangalore

2014-03-06 Thread Gora Mohanty 
On 6 March 2014 14:31, Thejesh GN i...@thejeshgn.com wrote:

 Thanks a lot Shree. I will get in touch with them.


 Just to update the group. I did a small POC recently to see if its 
 achievable.  I think its quite possible even-though a little difficult.

[...]

Yes, ODK is great.

 Some issues that I faced
 1. GPS logging is not very accurate. It has 4 to 8 meters of error, how do we 
 fix it?

4-8m is optimistic, and assumes quite some time spent to get a
proper GPS fix. Given the quality of the GPS in most phones, a
RMS error of 20m is more realistic. The only way to improve that
is to take a large number of readings, and average over these, but
this cannot be done with ODK out of the box. As the error in the
mean decreases as the square root of the number of observations,
one needs 25 readings to get from 20m to 4m, and 400 to get to
1m. However, if the aim is to identify the general locality, the default
GPS accuracy should be fine.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

2014-03-02 Thread Gora Mohanty 
On 2 March 2014 13:13, Thejesh GN i...@thejeshgn.com wrote:

 On Mar 1, 2014 7:59 PM, Gautam John gkj...@gmail.com wrote:

 On Sat, Mar 1, 2014 at 7:34 PM, Gora Mohanty g...@mimirtech.com wrote:

  idea of having a way to get data from the Excel files, and other
  sources, on
  data.gov.in and other places, and providing real API-based access, and
  have
  some thoughts on this. Should we discuss this here, or take it offline.

 Wouldn't something like CKAN work?

 http://docs.ckan.org/en/latest/api/index.html

 I had not checked the CKAN in a long time. It looks very interesting. It
 doesn't say anything about GIS part. I am very interested in GIS. I will
 probably install over the weekend and let group explore.

We have extensively used GIS with GeoDjango. While GIS is doable,
maybe we should agree upon what the primary purpose of the site should
be. What I thought that you were proposing was a site that acts as a
central location where data from multiple Indian data sources are accumulated,
and provided to users via an API.

While we are also interested in GIS, visualisation, and reporting, to my mind
that should be a secondary focus for this site.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

2014-03-02 Thread Gora Mohanty 
On 2 March 2014 14:27, Thejesh GN i...@thejeshgn.com wrote:
[...]
 My focus is to deliver bangalore data through API. that includes location
 based query too
 - For example given lat, long what are the nearest bus stops.
 - Given a lat, long what are the nearest schools
 - Given a lat, long whats the AC, PB and WARD
 etc

 There are the things that can be consumed by an app directly.

OK, that makes sense. GIS queries are expensive though, and we
will have to worry about how best to handle these.

If you can check out CKAN, I will start articulating our ideas on
how to crowd-source data-import from poorly-structured CSV/Excel
files which unfortunately seems to be the preferred format for many
data sites.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

2014-03-01 Thread Gora Mohanty 
On 1 March 2014 19:27, Thejesh GN i...@thejeshgn.com wrote:


  Let me know if you want to contribute.

 What are you going to use to build the API? We would be interested if
 Python/Django could be used. We might also be able to share in the
 hosting, and would probably do that in the AWS-Singapore region.


 Yes. In fact http://geodjango.org/ and back end will be PosGIS and
PostgresSQL. Can be easily hosted on AWS EC2.

Sounds good, though I think that we should first focus on building the API,
with GIS features coming later. We have been internally kicking around the
idea of having a way to get data from the Excel files, and other sources,
on data.gov.in and other places, and providing real API-based access, and
have some thoughts on this. Should we discuss this here, or take it offline.

My concern with things like this remains the lack of clear licences for the
usage of data from data.gov.in . It would be terrible to build a whole
system, and then have the data tap turned off.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

2014-03-01 Thread Gora Mohanty 
On 1 March 2014 19:59, Gautam John gkj...@gmail.com wrote:
 On Sat, Mar 1, 2014 at 7:34 PM, Gora Mohanty g...@mimirtech.com wrote:

 idea of having a way to get data from the Excel files, and other sources, on
 data.gov.in and other places, and providing real API-based access, and have
 some thoughts on this. Should we discuss this here, or take it offline.

 Wouldn't something like CKAN work?

 http://docs.ckan.org/en/latest/api/index.html

Thanks for sharing this. It looks interesting, and at least at first glance
would seem to be a good fit for basing the site upon.

There is still a challenge in getting data from disparate, unstructured,
data sites in India into such a CKAN site.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

2014-03-01 Thread Gora Mohanty 
On 2 March 2014 08:40, Nagarajan M mnagarajan...@gmail.com wrote:

 Are there any open source applications already that perform similar work of 
 aggregating data from different sources and formats and provide it in a 
 smooth API.

None that I know off of the top of my head, though we will look
before starting to roll our own. There are applications that can
help with bits and pieces of this task, but not an overall solution.


 I have different applications that work on different platforms and 
 technologies and this kind of API can help in acting as a one mega database 
 server for all apps.

 Atleast it can act a data repository for MIS and analysis for BI.

Yes, I agree that this would be very useful. Maybe it can also
convince government, and other institutions, in India to have
proper licences for the data that they release.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

2014-02-28 Thread Gora Mohanty 
On 1 March 2014 13:16, Thejesh GN i...@thejeshgn.com wrote:
 I am planning to build an API to deliver data from openbangalore.com

 It would live at
 api.openbangalore.org

 Some interesting apis are

 http://data.london.gov.uk
 Sunlight's  APIs
 Etc

 Suggest me if you know any city APIs thats working really well,  that can be
 our role model.

 Aim is to provide free api aceess (as much bandwidth as i can afford) for
 people who dont want to download data again n again for updates.

 Let me know if you want to contribute.

What are you going to use to build the API? We would be interested if
Python/Django could be used. We might also be able to share in the
hosting, and would probably do that in the AWS-Singapore region.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Tree survey in bangalore

2014-01-10 Thread Gora Mohanty 
Hi,

Sorry for top-posting, but on my phone. I agree that ODK makes sense for
this. As mentioned earlier, we already have a platform, Fieldata, with a
web interface, where the data collection part is built on top of ODK. We
can offer this both free of cost as well as without restrictions on
extracting the data. Also would be glad to help out in setting up the
forms. Please let me know.

Regards,
Gora
On Jan 10, 2014 4:12 PM, Thejesh GN i...@thejeshgn.com wrote:

 I have set up ODK at

 https://openbangalore.appspot.com

 Will try to map some tress on my street this weekend, I will keep you guys
 updated.


 Note: ODK seemed like the most easy and practical way to do it.



 Thej
 --
 Thejesh GN *⏚* ತೇಜೇಶ್ ಜಿ.ಎನ್
 http://thejeshgn.com
 GPG ID :  0xBFFC8DD3C06DD6B0


 On Sun, Jan 5, 2014 at 3:04 PM, Gora Mohanty g...@mimirtech.com wrote:

 On 5 January 2014 13:34, rajesh sha...@gmail.com wrote:
  I think its a great idea to map each tree, actually each sapling, but go
  beyond a simple count.
 
  What is important when we hear stories like 50,000 trees were planted
 is
  to be able to followup and see if the survival rate was 1% or 0.01% or
  0.0001%
 
  If we map every tree we should see how long each tree lives, so we can
 map
  how many are being cut. If not, what is the growth and condition (leaves
  covered with dust). Is it concretized at the base or is there soil and
 air.
  Possibly a very depressing picture.
 
  Lets not worry about productizing or revenue.

 We have a mobile-based data collection platform, Fieldata
 ( http://www.fieldata.org ) that can be used for this, free of
 cost at the basic tier which should suffice here. One builds
 a form for the data collection, using our GUI form builder,
 which contains all details necessary. The data can include
 GPS locations, photos, etc. The form is downloaded to an
 application on an Android phone, based on OpenDataKit,
 and anyone with access can submit data. We would also be
 glad to help in building the form.

 To get a quick picture of the facilities available on Fieldata,
 log in to a demo account from the top right of the home page:
 User/password = gora+fossdo...@mimirtech.com/foss.in123
 On the dashboard that you arrive at, choose the Demo
 questionnaire, and you can have various views of the data
 from under the Reports  Data drop-down towards the top.
 Data can be exported as CSV.

 Regards,
 Gora

 --
 For more details about this list
 http://datameet.org/discussions/
 ---
 You received this message because you are subscribed to the Google Groups
 datameet group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to datameet+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


  --
 For more details about this list
 http://datameet.org/discussions/
 ---
 You received this message because you are subscribed to the Google Groups
 datameet group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to datameet+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Live Bangalore weather info from personal weather stations

2014-01-02 Thread Gora Mohanty 
On 1 January 2014 22:54, Pradeep Mohandas pradeep.mohan...@gmail.com wrote:
 hi,

 The IMD has these AWS - Automated Weather Stations, data from which is also
 available - http://www.imdaws.com/ViewAwsData.aspx - these seem to be in
 addition to the data available from the various met stations. Would be
 interesting to compare data from the personal and automated weather
 stations?
[...]

Does this work for you? Just tried the only station in Delhi, and
I do not get any data from View data even with all available dates
in Dec. 2013 selected. Download data wants one to log in.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Web Logs for the browsers

2013-12-13 Thread Gora Mohanty 
On 13 December 2013 15:55, Santosh Kumar san.kumar...@gmail.com wrote:
 Hi,

 How do we get access to the web logs of the browser. Is there any open
 source platform to get it.
[...]

What do you mean by web logs of the browser. A user's browser
does not normally keep logs.

Do you mean logs from a site's webserver? If so, you can get these
only for sites that you control, in a location dependent on the site
configuration. It would be a huge security hole if anyone could get
webserver logs for any site.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Web Logs for the browsers

2013-12-13 Thread Gora Mohanty 
On 13 December 2013 18:20, Santosh Kumar san.kumar...@gmail.com wrote:
 Thank you for your reply Gora. I mean from the site's web browser. For
 example, my web history logs in the google for the day.
[...]

This is still not making sense to me. There is no such thing as a site's
web browser, so I presume that you mean the web server. I do not then
understand what is my web history logs in the google for the day. As
mentioned earlier, webserver logs for external sites are not normally made
accessible to the browser, or to Google.

If it is your own site, you can add some kind of analytics for users visiting
the site: Probably the easiest way to start is with Google Analytics. Also,
awstats is an open-source logfile analyser that can deal with Apache logs.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] how to crawl Indian newspaper sites

2013-12-10 Thread Gora Mohanty 
Hi,

That problem is common to whatever crawler you use. Nutch will extract
links from JavaScript, but that's it. I would use Rhino, and a custom
HtmlParser plugin for Nutch. This is admittedly non-trivial, but I know of
no open source tool that already does this.

Regards,
Gora
Hi Gora,

The problem with using scrapy (or just simply BeautifulSoup) is that some
newspapers generate content dynamically, using javascript. The possible
solutions we found was using phantomjs or Goose (a python library). If
Nutch can handle content generated through javascript (which it doesn't
appear to) then we'll use it.

Debamitro
On Tuesday, 10 December 2013 11:41:03 UTC+5:30, Gora Mohanty wrote:

 On 9 December 2013 19:37, Debamitro Chakraborti deba...@gmail.com
 wrote:
  Any way to crawl the back issues of prominent Indian newspapers like The
  Hindu, TOI, Indian Express, Hindustan Times etc?
  I was part of a team which needed to analyse news reports from a time
 frame
  and we hacked together a TOI crawler (which still has limitations) and
 were
  working on a The Hindu crawler -- would love to know about something
 simpler
  that is already available.

 The newsrack.in recommendation is a good one, but Newsrack
 is intended as much more than a simple crawler. If a crawler is
 what you need, you should look into something like Nutch
 ( http://nutch.apache.org/ ). If you prefer to write your own for
 simple, non-generic. needs we have happily used Scrapy in
 the Python world ( http://scrapy.org/ ).

 Regards,
 Gora

 --
For more details about this list
http://datameet.org/discussions/
---
You received this message because you are subscribed to the Google Groups
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an
email to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] [datameet.delhi] next datameet-up in delhi 22 nov?

2013-11-19 Thread Gora Mohanty 
On 20 November 2013 10:15, sumandro suman...@gmail.com wrote:

 Dear all,

 Sorry for the short notice.

 Can we have a DataMeet-Up on Friday, November 22, evening (say 5:30 pm)?

 Also, Isha+Amitangshu, can we organise the meet-up at Akvo office on the same 
 day?

The date and venue work for me.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] IWP: Data Capacity Building Case Study

2013-10-29 Thread Gora Mohanty 
Hi,

At the risk of tooting our own horn, we have built a system for such
smartphone/SMS based data collection, based on OpenDataKit, at
http://www.fieldata.org . Anyone can register at the site, and the base
tier is free of cost.

Please do note the disclaimer that Fieldata is a commercial venture, and I
have a stake in it.

Regards,
Gora
On Oct 29, 2013 8:57 PM, Samuel Rajkumar aadva...@gmail.com wrote:

 Hi Nisha,

 Though 'spreadsheets are ubiquitous', I would think a better way of data
 gathering would be through an Android survey tool, given that smartphones
 are only slightly less ubiquitous than spreadsheets. Open Data Kit is
 probably a good free option.

 Cheers,


 On Tue, Oct 29, 2013 at 12:36 PM, Nisha Thompson nisha.thomp...@gmail.com
  wrote:

 Hello All,

 I have been working with the Keystone Foundation, in the Niligiris on how
 they collect and manage data.

 We have been putting together a case study on how water related NGOs in
 the field deal with data and how they manage it.  We decided to document
 why data is important to them, how it influences their work, and what
 problems they encounter.  We also are offering some solutions and tips to
 basic problems.

 The first three posts of the series is up. The first two are background
 on Keystone and their projects. The 3rd is a post on tips for using excel
 more effectively for data management and analysis.


 http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-i

 http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-ii

 http://www.indiawaterportal.org/news/tips-working-spreadsheets-data-entry-storage-and-retrieval

 This series will continue over the next few months.  If anyone has any
 other tips or suggestions for excel or any other low cost, low tech
 solutions that could be worth exploring, please let me know!

 Any comments welcome!

 Nisha

 --
 Nisha Thompson
 Mobile: 962-061-2245

  --
 For more details about this list
 http://datameet.org/discussions/
 ---
 You received this message because you are subscribed to the Google Groups
 datameet group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to datameet+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


  --
 For more details about this list
 http://datameet.org/discussions/
 ---
 You received this message because you are subscribed to the Google Groups
 datameet group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to datameet+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] IWP: Data Capacity Building Case Study

2013-10-29 Thread Gora Mohanty 
On Oct 30, 2013 10:13 AM, Samuel Rajkumar aadva...@gmail.com wrote:

 Respectfully disagree, Nisha.

 There really is no problem with spreadsheets being used to analyze data.
But collecting data with paper forms and then typing it into the
spreadsheet is just very inefficient and error prone.
[...]

Agreed. Our experience has been that most organisations are immediately
appreciative of the benefits of data collection based on smartphones. As
you note, Android smartphones are now cheap, and even with the one-time
sunk cost of purchasing the devices, our platform is usually cheaper than
paper-based methods, besides being less error-prone. SMS can be used if
there is a big sensitivity to price.

Spreadsheets are also unstructured, and difficult to convert to API-based
access, which is needed for any generic analysis methods not customised to
a specific data format embodied in the spreadsheet. IMHO, this is the
biggest problem with the current data sets on data.gov.in

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Twitter API 1.1

2013-09-03 Thread Gora Mohanty 
On 4 September 2013 09:31, sandeep khurana.sand...@gmail.com wrote:

 Hi folks,

 Looking for people who have worked on twitter API 1.1 or have good knowledge
 of python scripting or can lend access to gnip/datasift/topsy.

Are you looking for a commercial engagement? If so, please mark
such messages as such in the subject line, e.g., with a [Commercial]
tag, and invite responses outside of the mailing list.

We would be interested, and I will contact you off list.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Twitter API 1.1

2013-09-03 Thread Gora Mohanty 
On 4 September 2013 10:00, Khurana.Sandeep khurana.sand...@gmail.com wrote:
 Thanks, Gora.
 No, this is not commercial but for assistance with academic research.

OK. Would you like to describe the project in
more detail, and/or present it at the next meeting?

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] [datameet.delhi] invitation to next datameet-up in delhi, saturday, 31st aug, 5:30 pm

2013-08-25 Thread Gora Mohanty 
On 25 August 2013 11:00, sumandro suman...@gmail.com wrote:
 Dear all,

 Am very happy to invite you all to the next Delhi DataMeet-Up on Saturday,
 31st August 2013, 5:30 pm, at Mimir Technologies office in Green Park
 Extension.
[...]

Here is the complete address, and a map location:

Fieldata
Basement, U-1 Green Park Extension
http://goo.gl/maps/OrH0k

The office is located just behind the main road that
goes by the Green Park metro station. The nearest
Metro station is Green Park. Please contact me at
9868527992 in case of any issues with finding the
place.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] [datameet.delhi] next datameet-up in delhi

2013-08-19 Thread Gora Mohanty 
On 19 August 2013 15:30, sumandro suman...@gmail.com wrote:

 In that case, how about organising the Delhi DataMeet-Up this weekend, on 
 Saturday 24th 5:30 pm?

 Is that too soon?

 Gora, can we organise it at your office on 24th?

Unfortunately, I am out of town on the 24th.

 Also, it would be great to partner up with the R users' group. But maybe it 
 would be more effective to do that after the August meeting, since in this 
 meeting we expect to set our own agenda and plan of action.

On the other hand, I think that most FOSS user groups
in India are so weak and fragmented that it is always
better to try and bring them together. The R User's meeting
is already set for the afternoon of the 31st at our office,
but we can stagger the times if you would like.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] [datameet.delhi] next datameet-up in delhi

2013-08-18 Thread Gora Mohanty 
On 18 August 2013 20:38, sumandro suman...@gmail.com wrote:
 Mithila, August 31st at 5 pm works well for me too.

 Gora, can we organise the meeting at your office on that day?
[...]

Yes, let's do that. Also, we have a R User's group
that will be meeting that day. I would suggest that
we could fruitfully combine the two meetings. Let
me consult them, and I will post an announcement
in the next couple of days.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Converting pdf to csv?

2013-07-01 Thread Gora Mohanty 
On 1 July 2013 12:16, Debamitro Chakraborti debami...@gmail.com wrote:
 Hi,

 Is there any way to convert the table in the attached pdf's into csv? They
 are voter lists from a govt website and a friend needs this badly (he is
 ready to pay for this as well).
[...]

What do you want to extract from these? Unfortunately, the
Bangla text is not in Unicode, and thus it would take significant
time and effort to convert that. However, if you can describe
which of the numeric values to extract, that can probably be
done.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] CorporateDir

2013-06-30 Thread Gora Mohanty 
On 30 June 2013 21:15, Gautam John gkj...@gmail.com wrote:
 Anyone know who runs http://corporatedir.com/

whois corporatedir.com lists the domain registrant as
   Kumar, Karan karanku...@abhinav.me
   A Bizzbay Building
   Khondwa
   Pune, Maharashtra 411020
   IN
   9872699124

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Population density map for BLR

2013-06-30 Thread Gora Mohanty 
On 29 June 2013 05:42, Tejas Pande tejaspa...@gmail.com wrote:
 Hello. I am looking for the most recent population density map for the city
 of Bangalore. Any help is greatly appreciated.

If you have the data---at least counts and locations---producing
a map is easy. Ideally, the data would also include boundaries
for administrative areas in Bangalore, but it might be possible
to find that information from other sources.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Data Repository API for Government

2013-06-30 Thread Gora Mohanty 
On 1 July 2013 09:28, Nagarajan M mnagarajan...@gmail.com wrote:
 Hi All,

 The Open Data movement is evolving rapidly with the publication of datasets
 on www.data.gov.in

 The problem with government data is that its more statistics than actionable
 data. However the open data movement will evolve to demand for more granular
 and near real time data.

 The Government system will not be able to provide it because the systems of
 data collection and management are not standardized. There are not enough
 tools that are built to enable the Government machinery to integrate data
 collection through work flow.

Actually, I think that it will *have* to be done through the
Government of India, and the people running data.gov.in
It is just too difficult for third-party developers to do this
in a consistent manner that will keep working in the future:
Please see below.

 I am interested in creating a Data Repository that can act as an API where
 different workflows of government can link up and operate. Use existing
 metadata and also add their own.

 I request your suggestions with regard to architecture, technologies and
 tools.
[...]

Architecture, tools, etc., are not that complicated. One
could start with a system that parses the current spreadsheets,
digests the data, and provides an API. However, there are
several issues with third parties doing this:
1. While several people have done specific case studies by
manually extracting the data from the spreadsheet files,
this is obviously not something that can scale.
2. It is possible to build a system that enlists the help of people
interested in specific datasets. For example, a prototype of
such a system would parse the first few lines of any given
spreadsheet file, make an educated guess about data types,
and allow the user to modify the guesses.
3. Such crowd-sourcing might work at one level, but a big
problem is that:
(a) There is no consistent standard followed by the spreadsheets.
 E.g., at least names of states, and date formats could
 be standardised.
(b) Several of the spreadsheets that we looked at have
 internally inconsistent data. Please see the issues
 brought up by Supreet at
 http://www.mail-archive.com/ilugd@lists.linux-delhi.org/msg29943.html
4. Finally, there is the question of what incentive there is for
third-party developers to build such an API?
   For sustainability, this either has to be done by a publicly-
funded institution or there has to be a commercial basis for it.
If it is done through public funds, why should data.gov.in not
handle it themselves?
   There might well be a business case in selling access to such
an API, but the cost of developing the API and hosting the
services will be quite high. However, I cannot seem to find any
indication on the data.gov.in sites as to whether such third-
party, commercial access is allowed. In fact, I could not find
any information on the terms of usage of the data published
there.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Data Repository API for Government

2013-06-30 Thread Gora Mohanty 
On 1 July 2013 10:16, Venkata Pingali ping...@gmail.com wrote:
 I broadly agree. Technology is the easy part. I would think in terms
 of architecture of coordination. Let me comment on one related
 aspect.

 Finally, there is the question of what incentive there is for
 third-party developers to build such an API?

 Surprisingly enough, it doesnt have to be very high. The process has to be
 efficient. If it requires too much of mental and process context switch, it
 gets difficult. People like me are not in school anymore.
[...]

Not sure what you mean by this response. The process
of what?

A third possibility exists besides the two that I mentioned
in my earlier message: That of having an open-source
group that does this, where the incentive for doing the
development for free is recognition. However, the hosting
costs will still be high. My opinion also is that given the
current strength and abilities of open-source communities
in India, even development in this manner will be very
difficult.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] Data Repository API for Government

2013-06-30 Thread Gora Mohanty 
On 1 July 2013 10:28, Venkata Pingali ping...@gmail.com wrote:
 I meant coordination process. Many tasks involving data (esp collection)
 tend to be time consuming. I did a bunch of that in my previous life
 (lots of calls, emails, travel, campaigns etc.). It is hard to do it now.

Agreed. As you mentioned, and as I also believe, the technology
is the relatively easy part.

 BTW, I didnt imply that there needs to be monetary incentive, even in
 industry. I believe that open data is a reward in itself.

Well, here we will have to agree to disagree.

IMHO, this particular problem is of a scale and complexity that
cannot easily be addressed in a voluntary context. One might be
able to build a prototype, or even a working system, using open-
source volunteers working free of cost (personally, I think that
even this will be very difficult given the current status in India).
However, sustaining it is not something that can be done without
significant funding. Just the hosting costs will run into at least a
couple lakhs/year at any reasonable scale.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
datameet group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.