Julian Liverpool appears to use Modern.gov as its CMS, so adding this to theyworkforyou_local should be a 5 minute job (the scraper/parser is already written). I also added this morning api interface to return meeting data for given council via html/XML/json. What it doesn't do, however, is revisit the page to check for updates yet. That's on my to do list.
Contracts is a good one to deal with. Doing one for the London councils is fairly straightforward (in theory anyhow, they're centralized at LondonCouncils, though somehow I would expect there to be more than are shown on there), but I suspect other local authorities will require individual conversations, and it would be great to get a community going for that. Any ideas gratefully received. Cheers C Julian Todd wrote: > I know a lot about Liverpool City Council. I've got a project to > webscrape the meeting calender and produce alerts on this: > > http://councillors.liverpool.gov.uk/mgCalendarMonthView.aspx?GL=bcr=1 > > I'm also attempting to get the contracts the city has signed with suppliers. > > More later, > > Julian. > > > On Wed, Jun 17, 2009 at 12:46 PM, Tim Duckett <[email protected]> wrote: > >> This is interesting - there's a small group of us doing something >> similar with (or should that be *to*?) Sheffield City Council, and >> it's a nightmare. Where meeting minutes exist online they're >> published as Word docs, and they're written in a "narrative" style - >> for all its faults, Hansard is a world away from local government when >> it comes to scrapeable transparency. So far we've been working on >> indexing the docs and presenting that content alongside search results >> from the main SCC site. It's not TWFY, but at least it provides a >> better way of exposing individual activity that would otherwise be >> buried away in documents. >> >> Does anyone have any experience of managing to persuade LAs towards >> presenting the source information in more usable formats? >> >> On 16 Jun 2009, at 14:14, CountCulture wrote: >> >> >>> Quick note about something I've been working on in my spare time: >>> >>> http://theyworkforyoulocal.com -- a small app to scrape and parse >>> local >>> authority info. >>> >>> At the moment, it's barely more than a proof of concept, with only >>> about >>> 20 or so councils parsed, and even then only current councillors, >>> committees, committee membership and forthcoming meetings are parsed. >>> >>> On the upside, it's fairly quick for me to add new parsers for >>> councils >>> (and reuse ones already written if they use same CMS), there's an API >>> built in (basically just add .json or .xml to get the info as json or >>> XML), and there's lots of potential. >>> >>> Getting this far has also been an education in understanding what a >>> full-blown twfy_local might look like (in general there seems no way >>> to >>> see how councillors voted, for example), the need for such a resource >>> (there's no publicly available central repository for council election >>> results, for example), and the sorry state of local authority websites >>> (just finding a list of councillors is a challenge on some, and don't >>> get me started on the HTML markup). >>> >>> Comments welcome. Code is at >>> http://github.com/CountCulture/twfy_local_parser/ (I'll probably GPL >>> it >>> soon). Bug reports at >>> http://github.com/CountCulture/twfy_local_parser/issues and offers of >>> help to countculture at googlemail dot com. >>> >>> I'd especially be interested in hearing from anyone who's got any >>> knowledge about local authority CMSs (e.g. there seem to be several >>> different versions of Modern.Gov producing different URLs), or sources >>> for more data other than the local authority websites (e.g. eGR, >>> info4local). >>> >>> Cheers >>> >>> C >>> >>> _______________________________________________ >>> Mailing list [email protected] >>> Archive, settings, or unsubscribe: >>> https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public >>> >> --- >> Tim Duckett >> +44 (0) 7525 786 492 >> >> >> _______________________________________________ >> Mailing list [email protected] >> Archive, settings, or unsubscribe: >> https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public >> > > > _______________________________________________ Mailing list [email protected] Archive, settings, or unsubscribe: https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public
