Murray, Simon (IED) wrote:
I'm interested in using .Net and the HttpWebRequest class, but would welcome any guidance on the subject particularly when accessing data spanning across multiple pages.
http://www.crummy.com/software/BeautifulSoup/ might be useful? I've heard good things about it.
Our parser of Hansard (for which we have a licence, I should point out) has to cope with things spanning pages. It used to just look for the "Next Section" link and follow that until they stopped, but these are occasionally missing, so it now stores all the links from an index page, starts following Next Section links and hopefully works out what to do if one is missing.
-- ATB, Matthew - Sent via the backstage.bbc.co.uk discussion group. To unsubscribe, please visit http://backstage.bbc.co.uk/archives/2005/01/mailing_list.html. Unofficial list archive: http://www.mail-archive.com/[email protected]/
