Other than "respect robots.txt and don't make 5,000 requests/second", does anyone know if there are any rules about scraping websites and performing some form of analysis on the aggregate results? I'm particularly thinking of any legal obstructions which prevent people from aggregating data, which presumably Google, Yahoo etc. (and mySociety in some cases I guess) manage to get around or ignore.
Cheers, Paul -- Paul Waring http://www.pwaring.com _______________________________________________ Mailing list [email protected] Archive, settings, or unsubscribe: https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public
