Hi there, Cool - great to see people hacking on councils, it's been something I've wanted to see for ages.
I see you've gone straight for getting the councillors of several different councils, but I'd actually suggest going deeper rather than wider. Why not just dive deep into one council and see if you can get transcripts or other documents nicely scraped and parsed? I'd love to see at least a handful of councils in TheyWorkForYou proper by the end of the year. Well done anyway! Tom 2009/6/16 CountCulture <[email protected]>: > Quick note about something I've been working on in my spare time: > > http://theyworkforyoulocal.com -- a small app to scrape and parse local > authority info. > > At the moment, it's barely more than a proof of concept, with only about > 20 or so councils parsed, and even then only current councillors, > committees, committee membership and forthcoming meetings are parsed. > > On the upside, it's fairly quick for me to add new parsers for councils > (and reuse ones already written if they use same CMS), there's an API > built in (basically just add .json or .xml to get the info as json or > XML), and there's lots of potential. > > Getting this far has also been an education in understanding what a > full-blown twfy_local might look like (in general there seems no way to > see how councillors voted, for example), the need for such a resource > (there's no publicly available central repository for council election > results, for example), and the sorry state of local authority websites > (just finding a list of councillors is a challenge on some, and don't > get me started on the HTML markup). > > Comments welcome. Code is at > http://github.com/CountCulture/twfy_local_parser/ (I'll probably GPL it > soon). Bug reports at > http://github.com/CountCulture/twfy_local_parser/issues and offers of > help to countculture at googlemail dot com. > > I'd especially be interested in hearing from anyone who's got any > knowledge about local authority CMSs (e.g. there seem to be several > different versions of Modern.Gov producing different URLs), or sources > for more data other than the local authority websites (e.g. eGR, > info4local). > > Cheers > > C > > _______________________________________________ > Mailing list [email protected] > Archive, settings, or unsubscribe: > https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public > _______________________________________________ Mailing list [email protected] Archive, settings, or unsubscribe: https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public
