hello and good evening new to scrapy and to this list.
i have seen the python scraper - described here: http://blog.ouseful.info/2010/11/03/accessing-government-education-data-in-scraperwiki-via-the-edubaseeducation-datastore-api/ well this is very interesting. I am not a programmer - and therefore some of the techniques and ideas of your scraper are too complicated to me. But it seems to be the tool of choice: i need a scraper that feteches all the data of south amerian schools - eg. via wget of planet-files of openstreetmap. i am a teacher - and i work on a little project in the field of education so to describe my goals - i want to say some words and thoughts regarding the philantropic impact i think that we should use openstreetmap to: - improve the lives of poor children and youth by providing basic necessities, critical benefits and services, programs that support long-term development and opportunities that prepare them for success. we should take this into account to identify the places where schools and education programms need help., i want to use openstreetmap to identify schools with some of the following needs; - do they need comprehensive youth programs? - such that are designed to position youth for employment and help them become self-reliant adults and contributing members of their communities - this should be accomplished by focusing on the following areas. Education;:The surest path out of poverty is education. Unfortunately, education is sometimes considered a luxury by impoverished families, - i think that we should support basic education by providing benefits such as school supplies, fees and uniforms. - Additionally, we should help qualified youth complete secondary school and attend vocational training, colleges and universities so there fore i love to see some efforts in the osm-community. a scraper for all south american schools that gives back the data in a mysql db would be great. If i would have one scraper-code then the diffs and updates would be quite simple - i think? i love to hear from you best whishes martin btw. at the moment i try to do different requests- all with focus on schools; if you whish to help me - i would be happy motto: We are driven by the belief that, collectively, we can solve even the most difficult problems our country is facing. Some ideas regarding the aimed goals: i want to store the results in a mysql-db note: it is only necessary to have straightforward from files that - no backimport of data to OSM or to whole maps. My interest lies in the data of the schools like Name, Adress, Country, Town and postal code, Website etc. etx. - That are only textual information. Probably we can do this with a mysql-Db - or do we need a PostgreSQL? again: i want to gather data from .o5m & pbf file - several ways are possible: what is wanted: i look for schools - with all of the following tags: all i want is to get POI out of the OSM-File - and to hold them on a dayly updated version: in other words - i want to keep them updqated - day by day. Well i guess that i can do that with osmfilter and osmconvert - but there are probably better ways to do it: A python-scraper would be great. - PostGis as a solution for getting a database populated by osm2pgsql or osmconvert / -filter to create csv-sheets: probably PostGIS is tooo big and the learing curve is to steep; Note;: i only want to have POIS from the OSM-File. Nothing more. - MySQL seems to be better for what i am having in mind Technologically and methodolocically i am inspired by Martijn van Exel approach to keep an overview on the US-bridges note;: he looks for the bridges in the US ( http://oegeo.wordpress.com/2012/03/06/a-self-updating-openstreetmap-database-of-us-bridges-a-step-by-step-guide/ ) But - Martijn wanted to have the map-data - i only need the textual data - that means that the database should not be the problem. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
