Hi, I'm wondering if anybody knows of any web-scraping frameworks in Node.JS?
Previously, there was node.io (https://github.com/chriso/node.io), however, the project was recently discontinued. Googling for Node.JS and web scraping, most of the guides online just talk about using requests and cheerio - it works, but you need to handle a whole bunch of things yourself (throttling, distributing jobs, configuration, managing jobs etc.). On the Python side, I know of Scrapy (https://github.com/scrapy/scrapy), which is using Twisted for asynchronicity On the Ruby side, Nokogiri (http://nokogiri.org/) is meant to be good, although I haven't dived into it much. Is there anything equivalent in the Node world? Or what approaches are people using to tackle this problem? Cheers, Victor -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
