On Fri, May 30, 2014 at 7:20 PM, Alan Gauld <alan.ga...@btinternet.com> wrote: > > >> If a site offers an API that returns the data you need then use it, >> If not you have few alternatives to scraping (although scraping >> may be 'illegal' anyway due to the impact on other users). But scraping, >> whether a web page or a GUI or an old mainframe terminal >> is always a fragile and unsatisfactory solution. > >Okay I think learning how to scrap (library or framework) is not worth >the trouble. Especially if some people consider it illegal. Thanks for >the input. > > >As I say, sometimes you have no choice but to scrape. Its only 'illegal' if the site owner says so, in other words if their terms of use prohibit web scraping. If they have gone to the effort (and cost) of providing an API then it probably means scraping is prohibited. But many (most!) sites don't offer APIs and most smaller sites don't prohibit scraping, so it is still a valid technique. But before you try its always worth checking whether an API exists and whether scraping is permitted.
And by 'illegal' I mean you are unlikely to be prosecuted in a court but you are likely to find your IP address and/or account closed. The systems generally monitor activity and if an account is navigating through pages too quickly to be a human they often close the account down. Alan g.
_______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
