On Mon, Oct 25, 2010 at 3:16 PM, David Jones <[email protected]> wrote: > On 24 October 2010 22:35, [email protected] > <[email protected]> wrote: > > Ladies and Gents, > > The plan for November's meeting is to get together for a Code Dojo! This > is > > an opportunity for Pythoneers and Pythonistas of all levels to flex their > > fingers and do some group coding with the aim of improving our coding > > skills. In my limited experience, no matter what your level, everyone > gets > > to learn something new while having a bit of fun at the same time. > > Did you discuss _what_ you might do at a Dojo? > I'm totally open to other suggestions, but here is one I've been > thinking about a bit (and that i would like to try and turn into a > larger project, so a "starter" kit / trial would be most welcome): > > It's basically scraping weather data to make it useful for climate > analysis. > > Scraperwiki.com would be interested to sponsor this idea with beer and pizza. We have been gathering some datasets weather already so it would be great if we could help your project. http://scraperwiki.com/tags/weather
We have also been doing some work on pair programming and at Europython discussed hosting a dojo type event where we could test this idea with some intested users. > One of my hats is Clear Climate Code: http://clearclimatecode.org/ . > The short version (go to the website for the longer version) is that > we have a program called ccc-gistemp that takes temperature data from > weather stations and turns it into a graph of global temperature > trends. > > The main source of data for this program is GHCN, but for various > reasons this dataset is sparse in some areas of the world where we > could do better. For example, in the UK, there are only 11 stations > with data for the last 10 years. There are hundreds of weather > stations in the UK. > > Often the data is available from websites and such, the challenge is > to get it into a form suitable for ccc-gistemp to use. > > A concrete example that we could work on in a Dojo is this FOI > request: > http://www.whatdotheyknow.com/request/weather_data_kielder_and_alice_h > > the data is in an excel file in a zip file. So ideally we would > create a Python script to: > - download the file > - unzip it > - open the excel file (xlrd, or something else?) > - reformat the data to GHCN format (which is very simple, each row has > a station identifier, a year, and 12 monthly temperature values). > > What do people think? > > (by the way, there are more places to scrape: Environment Canada make > their data available here using a very inconvenient web form: > http://www.climate.weatheroffice.gc.ca/Welcome_e.html ; I've already > started scraping some of the UK Met Office data from here: > http://www.metoffice.gov.uk/climate/uk/stationdata/ ) > > Cheers, > drj > > -- > To post: [email protected] > To unsubscribe: [email protected] > Feeds: http://groups.google.com/group/python-north-west/feeds > More options: http://groups.google.com/group/python-north-west > -- To post: [email protected] To unsubscribe: [email protected] Feeds: http://groups.google.com/group/python-north-west/feeds More options: http://groups.google.com/group/python-north-west
