Re: [datameet] IWP: Data Capacity Building Case Study
In Akshara's experience, because we devolve data collection to the teachers themselves and have no real control over them, giving them smartphones would be risky for us because we'd probably lose many of them. Also, with paper, the replacement cost and time is easy and it is a known work flow for most government institutions - stamps, seals, signatures and what not. Lastly, we currently have a error rate of paper to digital transcription under 0.5% because our data entry work flow uses a double entry system. Also, at the scale we operate at, phone devices are expensive and it's much easier to scale paper and data entry operators. On Wed, Oct 30, 2013 at 10:37 AM, Nisha Thompson nisha.thomp...@gmail.com wrote: We're not really disagreeing. I agree with you, but you can't just say they should use Android App! Technology adaption is still an issue with NGOs that work in communities, there are language issues (most data is collected in the local language on paper then translated), technology familiarity, and training and so on. Data management is still done with spread sheets and paper because that is what is easiest to train people with. You can say an Android is easy and cheap, but with a limited budget an NGO will say I'd rather spend the money on something else. The upcoming posts will deal with data standardization, and better ways to collect different types of data. In which tips and solutions technology related will be discussed in this context. Nisha On Wed, Oct 30, 2013 at 10:12 AM, Samuel Rajkumar aadva...@gmail.com wrote: Respectfully disagree, Nisha. There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone. Smartphones are now about 4k, much cheaper than a computer that can run Open Office / MS Office. Most survey apps take into account intermittent coverage and cache results locally till they can be uploaded, and so on. Historical data can sit on a spreadsheet for now, but new data can and should go into a properly normalized database, and then exported to whatever format is convenient. And then there's also the wonderful side effect of not having to version spreadsheets, as anyone with 2013_new_project_methodology_x_survey_results_final-new-updated-2.ods will tell you. Regards, On Wed, Oct 30, 2013 at 9:41 AM, Nisha Thompson nisha.thomp...@gmail.com wrote: Thanks guys! We are looking at teh experience local NGOs have with data, and I'm sorry to say technology is a huge issue. Either these devices are too expensive or there are coverage problems. So we are trying to look at many solutions across many technology fronts (including low tech). SMS data collection actually is more useful in a lot of places than an Android App. Whether we like it or not spreadsheets are how most data is collected. We can tout new technology all we like but it's still a ways to go. Also a lot of data that is has been collected is sitting in excel sheets or word docs or paper formats that have been collected in the past. This post has come from many experiences my team had with NGOs trying to clean up past data. The upcoming posts will look at new projects that require data collection and how we can help make it more streamline. I will add these tools to a list of resources though for further reference! Nisha On Wed, Oct 30, 2013 at 7:56 AM, Gora Mohanty g...@mimirtech.com wrote: Hi, At the risk of tooting our own horn, we have built a system for such smartphone/SMS based data collection, based on OpenDataKit, at http://www.fieldata.org . Anyone can register at the site, and the base tier is free of cost. Please do note the disclaimer that Fieldata is a commercial venture, and I have a stake in it. Regards, Gora On Oct 29, 2013 8:57 PM, Samuel Rajkumar aadva...@gmail.com wrote: Hi Nisha, Though 'spreadsheets are ubiquitous', I would think a better way of data gathering would be through an Android survey tool, given that smartphones are only slightly less ubiquitous than spreadsheets. Open Data Kit is probably a good free option. Cheers, On Tue, Oct 29, 2013 at 12:36 PM, Nisha Thompson nisha.thomp...@gmail.com wrote: Hello All, I have been working with the Keystone Foundation, in the Niligiris on how they collect and manage data. We have been putting together a case study on how water related NGOs in the field deal with data and how they manage it. We decided to document why data is important to them, how it influences their work, and what problems they encounter. We also are offering some solutions and tips to basic problems. The first three posts of the series is up. The first two are background on Keystone and their projects. The 3rd is a post on tips for using excel more effectively for data management and analysis.
Re: [datameet] IWP: Data Capacity Building Case Study
Thanks Samuel for the correction. On Wed, Oct 30, 2013 at 11:30 AM, Samuel Rajkumar aadva...@gmail.comwrote: Hi Sailendra, This is actually OMR - Optical Mark Recognition, which is much simpler and less error prone than OCR. Regards, On Wed, Oct 30, 2013 at 11:23 AM, Sailendra Mishra biot@gmail.comwrote: Hi Guys, Have you ever considered any OCR based survey? OCR based survey may avoid technical and financial constraints (Popular with all major examinations conducted in India). Sample here ( http://chemmunish.com/answer%20key%20+2%20Full%20Physical.JPG). It require one time investment on a good OCR machine (really not very expensive and less error-prone). If the machine is a constraint, little programming with a fast scanner will do the job. However, the success with OCR based survey really depends on the designing of the survey. Regards Sailendra On Wed, Oct 30, 2013 at 10:41 AM, Gora Mohanty g...@mimirtech.comwrote: On Oct 30, 2013 10:13 AM, Samuel Rajkumar aadva...@gmail.com wrote: Respectfully disagree, Nisha. There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone. [...] Agreed. Our experience has been that most organisations are immediately appreciative of the benefits of data collection based on smartphones. As you note, Android smartphones are now cheap, and even with the one-time sunk cost of purchasing the devices, our platform is usually cheaper than paper-based methods, besides being less error-prone. SMS can be used if there is a big sensitivity to price. Spreadsheets are also unstructured, and difficult to convert to API-based access, which is needed for any generic analysis methods not customised to a specific data format embodied in the spreadsheet. IMHO, this is the biggest problem with the current data sets on data.gov.in Regards, Gora -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Sailendra Prasanna Mishra Linkedin: http://in.linkedin.com/in/sailendraprasannamishra Twitter: https://twitter.com/sailendram Skype: Sailendra.prasannam Few people in this world who make mistakes in right direction. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Sailendra Prasanna Mishra Linkedin: http://in.linkedin.com/in/sailendraprasannamishra Twitter: https://twitter.com/sailendram Skype: Sailendra.prasannam Few people in this world who make mistakes in right direction. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [datameet] IWP: Data Capacity Building Case Study
Hi Data is still collected using pen and paper in most cases. The spreadsheet comes in when the data management part comes in. In most cases, electronic input devices can't straightaway replace this because of a number of factors 1. Data collectors may not be used to using computers/tablets etc.. Smart phone use with active internet usage is quite low. 2. Giving each data collector a device may not be possible given the costs involved. 3. NGOs typically don't have capacity to manage data, leave alone web based apps, tablets etc. Even if they all have android smartphones, someone needs to design a good database. This capacity is also quite rare. 4. Design of questionnaires is also often not done very well, so that you can't predict that the responses will always be from a predefined list etc. The fundamental issue is the capacity to handle data from design to analysis and presentation. The android tech part is one piece of the puzzle. I see the fieldata type of initiatives helping solve part of the problem. Data literacy is a more urgently needed input. I do agree that android devices and associated tech such as opendatakit have helped bring down the cost to a potential adopter. However, most NGOs are unlikely to have any budget allocation towards data collection devices, unless they do a lot of surveys. As Nisha has mentioned, this is a first post in a series. We will be covering other aspects such as technology adoption as well. If any of you has any already documented cases on using opendatakit or any other such app we could benefit from your sharing the experience. Regards Bala On 30 October 2013 09:41, Nisha Thompson nisha.thomp...@gmail.com wrote: Thanks guys! We are looking at teh experience local NGOs have with data, and I'm sorry to say technology is a huge issue. Either these devices are too expensive or there are coverage problems. So we are trying to look at many solutions across many technology fronts (including low tech). SMS data collection actually is more useful in a lot of places than an Android App. Whether we like it or not spreadsheets are how most data is collected. We can tout new technology all we like but it's still a ways to go. Also a lot of data that is has been collected is sitting in excel sheets or word docs or paper formats that have been collected in the past. This post has come from many experiences my team had with NGOs trying to clean up past data. The upcoming posts will look at new projects that require data collection and how we can help make it more streamline. I will add these tools to a list of resources though for further reference! Nisha On Wed, Oct 30, 2013 at 7:56 AM, Gora Mohanty g...@mimirtech.com wrote: Hi, At the risk of tooting our own horn, we have built a system for such smartphone/SMS based data collection, based on OpenDataKit, at http://www.fieldata.org . Anyone can register at the site, and the base tier is free of cost. Please do note the disclaimer that Fieldata is a commercial venture, and I have a stake in it. Regards, Gora On Oct 29, 2013 8:57 PM, Samuel Rajkumar aadva...@gmail.com wrote: Hi Nisha, Though 'spreadsheets are ubiquitous', I would think a better way of data gathering would be through an Android survey tool, given that smartphones are only slightly less ubiquitous than spreadsheets. Open Data Kit is probably a good free option. Cheers, On Tue, Oct 29, 2013 at 12:36 PM, Nisha Thompson nisha.thomp...@gmail.com wrote: Hello All, I have been working with the Keystone Foundation, in the Niligiris on how they collect and manage data. We have been putting together a case study on how water related NGOs in the field deal with data and how they manage it. We decided to document why data is important to them, how it influences their work, and what problems they encounter. We also are offering some solutions and tips to basic problems. The first three posts of the series is up. The first two are background on Keystone and their projects. The 3rd is a post on tips for using excel more effectively for data management and analysis. http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-i http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-ii http://www.indiawaterportal.org/news/tips-working-spreadsheets-data-entry-storage-and-retrieval This series will continue over the next few months. If anyone has any other tips or suggestions for excel or any other low cost, low tech solutions that could be worth exploring, please let me know! Any comments welcome! Nisha -- Nisha Thompson Mobile: 962-061-2245 -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more
Re: [datameet] IWP: Data Capacity Building Case Study
Hi Nisha, Though 'spreadsheets are ubiquitous', I would think a better way of data gathering would be through an Android survey tool, given that smartphones are only slightly less ubiquitous than spreadsheets. Open Data Kit is probably a good free option. Cheers, On Tue, Oct 29, 2013 at 12:36 PM, Nisha Thompson nisha.thomp...@gmail.comwrote: Hello All, I have been working with the Keystone Foundation, in the Niligiris on how they collect and manage data. We have been putting together a case study on how water related NGOs in the field deal with data and how they manage it. We decided to document why data is important to them, how it influences their work, and what problems they encounter. We also are offering some solutions and tips to basic problems. The first three posts of the series is up. The first two are background on Keystone and their projects. The 3rd is a post on tips for using excel more effectively for data management and analysis. http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-i http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-ii http://www.indiawaterportal.org/news/tips-working-spreadsheets-data-entry-storage-and-retrieval This series will continue over the next few months. If anyone has any other tips or suggestions for excel or any other low cost, low tech solutions that could be worth exploring, please let me know! Any comments welcome! Nisha -- Nisha Thompson Mobile: 962-061-2245 -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [datameet] IWP: Data Capacity Building Case Study
Hi, At the risk of tooting our own horn, we have built a system for such smartphone/SMS based data collection, based on OpenDataKit, at http://www.fieldata.org . Anyone can register at the site, and the base tier is free of cost. Please do note the disclaimer that Fieldata is a commercial venture, and I have a stake in it. Regards, Gora On Oct 29, 2013 8:57 PM, Samuel Rajkumar aadva...@gmail.com wrote: Hi Nisha, Though 'spreadsheets are ubiquitous', I would think a better way of data gathering would be through an Android survey tool, given that smartphones are only slightly less ubiquitous than spreadsheets. Open Data Kit is probably a good free option. Cheers, On Tue, Oct 29, 2013 at 12:36 PM, Nisha Thompson nisha.thomp...@gmail.com wrote: Hello All, I have been working with the Keystone Foundation, in the Niligiris on how they collect and manage data. We have been putting together a case study on how water related NGOs in the field deal with data and how they manage it. We decided to document why data is important to them, how it influences their work, and what problems they encounter. We also are offering some solutions and tips to basic problems. The first three posts of the series is up. The first two are background on Keystone and their projects. The 3rd is a post on tips for using excel more effectively for data management and analysis. http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-i http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-ii http://www.indiawaterportal.org/news/tips-working-spreadsheets-data-entry-storage-and-retrieval This series will continue over the next few months. If anyone has any other tips or suggestions for excel or any other low cost, low tech solutions that could be worth exploring, please let me know! Any comments welcome! Nisha -- Nisha Thompson Mobile: 962-061-2245 -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [datameet] IWP: Data Capacity Building Case Study
We're not really disagreeing. I agree with you, but you can't just say they should use Android App! Technology adaption is still an issue with NGOs that work in communities, there are language issues (most data is collected in the local language on paper then translated), technology familiarity, and training and so on. Data management is still done with spread sheets and paper because that is what is easiest to train people with. You can say an Android is easy and cheap, but with a limited budget an NGO will say I'd rather spend the money on something else. The upcoming posts will deal with data standardization, and better ways to collect different types of data. In which tips and solutions technology related will be discussed in this context. Nisha On Wed, Oct 30, 2013 at 10:12 AM, Samuel Rajkumar aadva...@gmail.comwrote: Respectfully disagree, Nisha. There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone. Smartphones are now about 4k, much cheaper than a computer that can run Open Office / MS Office. Most survey apps take into account intermittent coverage and cache results locally till they can be uploaded, and so on. Historical data can sit on a spreadsheet for now, but new data can and should go into a properly normalized database, and then exported to whatever format is convenient. And then there's also the wonderful side effect of not having to version spreadsheets, as anyone with 2013_new_project_methodology_x_survey_results_final-new-updated-2.ods will tell you. Regards, On Wed, Oct 30, 2013 at 9:41 AM, Nisha Thompson nisha.thomp...@gmail.comwrote: Thanks guys! We are looking at teh experience local NGOs have with data, and I'm sorry to say technology is a huge issue. Either these devices are too expensive or there are coverage problems. So we are trying to look at many solutions across many technology fronts (including low tech). SMS data collection actually is more useful in a lot of places than an Android App. Whether we like it or not spreadsheets are how most data is collected. We can tout new technology all we like but it's still a ways to go. Also a lot of data that is has been collected is sitting in excel sheets or word docs or paper formats that have been collected in the past. This post has come from many experiences my team had with NGOs trying to clean up past data. The upcoming posts will look at new projects that require data collection and how we can help make it more streamline. I will add these tools to a list of resources though for further reference! Nisha On Wed, Oct 30, 2013 at 7:56 AM, Gora Mohanty g...@mimirtech.com wrote: Hi, At the risk of tooting our own horn, we have built a system for such smartphone/SMS based data collection, based on OpenDataKit, at http://www.fieldata.org . Anyone can register at the site, and the base tier is free of cost. Please do note the disclaimer that Fieldata is a commercial venture, and I have a stake in it. Regards, Gora On Oct 29, 2013 8:57 PM, Samuel Rajkumar aadva...@gmail.com wrote: Hi Nisha, Though 'spreadsheets are ubiquitous', I would think a better way of data gathering would be through an Android survey tool, given that smartphones are only slightly less ubiquitous than spreadsheets. Open Data Kit is probably a good free option. Cheers, On Tue, Oct 29, 2013 at 12:36 PM, Nisha Thompson nisha.thomp...@gmail.com wrote: Hello All, I have been working with the Keystone Foundation, in the Niligiris on how they collect and manage data. We have been putting together a case study on how water related NGOs in the field deal with data and how they manage it. We decided to document why data is important to them, how it influences their work, and what problems they encounter. We also are offering some solutions and tips to basic problems. The first three posts of the series is up. The first two are background on Keystone and their projects. The 3rd is a post on tips for using excel more effectively for data management and analysis. http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-i http://www.indiawaterportal.org/articles/water-and-livelihoods-nilgiris-part-ii http://www.indiawaterportal.org/news/tips-working-spreadsheets-data-entry-storage-and-retrieval This series will continue over the next few months. If anyone has any other tips or suggestions for excel or any other low cost, low tech solutions that could be worth exploring, please let me know! Any comments welcome! Nisha -- Nisha Thompson Mobile: 962-061-2245 -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it,
Re: [datameet] IWP: Data Capacity Building Case Study
On Oct 30, 2013 10:13 AM, Samuel Rajkumar aadva...@gmail.com wrote: Respectfully disagree, Nisha. There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone. [...] Agreed. Our experience has been that most organisations are immediately appreciative of the benefits of data collection based on smartphones. As you note, Android smartphones are now cheap, and even with the one-time sunk cost of purchasing the devices, our platform is usually cheaper than paper-based methods, besides being less error-prone. SMS can be used if there is a big sensitivity to price. Spreadsheets are also unstructured, and difficult to convert to API-based access, which is needed for any generic analysis methods not customised to a specific data format embodied in the spreadsheet. IMHO, this is the biggest problem with the current data sets on data.gov.in Regards, Gora -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [datameet] IWP: Data Capacity Building Case Study
Hi Guys, Have you ever considered any OCR based survey? OCR based survey may avoid technical and financial constraints (Popular with all major examinations conducted in India). Sample here ( http://chemmunish.com/answer%20key%20+2%20Full%20Physical.JPG). It require one time investment on a good OCR machine (really not very expensive and less error-prone). If the machine is a constraint, little programming with a fast scanner will do the job. However, the success with OCR based survey really depends on the designing of the survey. Regards Sailendra On Wed, Oct 30, 2013 at 10:41 AM, Gora Mohanty g...@mimirtech.com wrote: On Oct 30, 2013 10:13 AM, Samuel Rajkumar aadva...@gmail.com wrote: Respectfully disagree, Nisha. There really is no problem with spreadsheets being used to analyze data. But collecting data with paper forms and then typing it into the spreadsheet is just very inefficient and error prone. [...] Agreed. Our experience has been that most organisations are immediately appreciative of the benefits of data collection based on smartphones. As you note, Android smartphones are now cheap, and even with the one-time sunk cost of purchasing the devices, our platform is usually cheaper than paper-based methods, besides being less error-prone. SMS can be used if there is a big sensitivity to price. Spreadsheets are also unstructured, and difficult to convert to API-based access, which is needed for any generic analysis methods not customised to a specific data format embodied in the spreadsheet. IMHO, this is the biggest problem with the current data sets on data.gov.in Regards, Gora -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Sailendra Prasanna Mishra Linkedin: http://in.linkedin.com/in/sailendraprasannamishra Twitter: https://twitter.com/sailendram Skype: Sailendra.prasannam Few people in this world who make mistakes in right direction. -- For more details about this list http://datameet.org/discussions/ --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
