Ian Forrester wrote, reordered slightly: > So what do you guys all think?
Sounds like a great idea; especially as I've been doing this for the BBC News front page (similar to the RSS feed, I guess) since 2005? :-) > I imagine within a few months, you could data mine out keywords and > trends. You certainly could. :) Perhaps with a tag cloud: http://www.dracos.co.uk/work/bbc-news-archive/tardis/tag-cloud/ Or watching how the word Iran is used over time: http://www.dracos.co.uk/work/bbc-news-archive/tardis/search-headline/?s=iran Or which pictures the BBC decides to illustrate stories about the Olympics: http://www.dracos.co.uk/work/bbc-news-archive/tardis/image-search/?s=olympics > It would also make a great archive of the news at the time Here's my archive of the front page of BBC News for the past 6 months or so: http://www.dracos.co.uk/work/bbc-news-archive/archive.php > Ideally in the future, you would all be able to add metadata around the > original items, setup unique queries and copy the whole database. Sadly, not sure I have the bandwidth for that. ;) You can create a unique timeline of how an event appeared on the front page: http://www.dracos.co.uk/work/bbc-news-archive/tardis/timeline/?t=Richard+Hammond%27s+crash > So how about we started to archive the RSS feeds into one large database, > index them by time and date, slap a nice REST interface on them and let > you guys have access to it? Seems odd for you to have to export the data from somewhere it should already be a database to RSS to import it back in again - perhaps public read access to the actual BBC News content management database would be a bit hard to justify. ;-) Apologies that my dataset doesn't have an API beyond what's there; I'd be happy to add things if people asked (no-one ever has :) ) and I had the time. But I can do database queries, e.g. what were the BBC News front page main headlines on New Years Day 2006? --8<----------------------- msbbc=> select url,headline,start,description from headline where date_trunc('day', start)='2006-01-01' and type='headline' order by start; /1/hi/uk/4570980.stm | Revellers ready to welcome 2006 | 2006-01-01 00:00:00 | Millions in the UK prepare to see in the New Year, while in London a Tube strike appears not to have caused chaos. /1/hi/uk/4570980.stm | Revellers welcome 2006 across UK | 2006-01-01 00:05:00 | Millions in the UK celebrate the New Year, while in London a Tube strike appears not to have caused chaos. /1/hi/uk/4570980.stm | Revellers welcome 2006 across UK | 2006-01-01 04:35:00 | Millions in the UK celebrate the New Year, with parties and spectacular firework displays in many cities. /1/hi/health/4529762.stm | Rift over greater hospital choice | 2006-01-01 13:40:00 | People in England are given a choice of hospitals for non-emergency surgery, but unions voice concerns. /1/hi/world/europe/4573572.stm | Ukraine gas crisis 'could hit EU' | 2006-01-01 16:20:00 | Gas exports to the European Union could be hit by the axeing of Russian supplies to Ukraine, officials say. /1/hi/world/europe/4573572.stm | Ukraine gas row hits EU supplies | 2006-01-01 17:40:00 | EU countries start to suffer knock-on cuts in gas imports as Russia axes supplies to Ukraine. /1/hi/world/europe/4573572.stm | Ukraine gas row hits EU supplies | 2006-01-01 17:45:00 | EU countries start to suffer knock-on cuts in gas imports as Russia axes supplies to Ukraine in a row over prices. --8<----------------------- Might be useful to someone, you never know! -- ATB, Matthew | http://www.dracos.co.uk/ - Sent via the backstage.bbc.co.uk discussion group. To unsubscribe, please visit http://backstage.bbc.co.uk/archives/2005/01/mailing_list.html. Unofficial list archive: http://www.mail-archive.com/[email protected]/
