On 02/24/2016 09:58 PM, Charles Curley wrote: > I have been using Perl's www:mechanize to scrape a series of web pages. > > Unfortunately the web page now includes some Javascript, which > mechanize does not handle. Suggestions? > > All of my code is in shell script and perl, so I'd like to stick with > those. > > Suggestions?
I haven no experience with it, but there's a framework called Selenium WebDriver (if I recall correctly) that drives a real browser to do screen scraping and automation. There are perl bindings for it, or rather a Perl client for it (which communicates over RPC with the WebDriver process). http://search.cpan.org/~aivaturi/Selenium-Remote-Driver-0.15/lib/Selenium/Remote/Driver.pm Seems a bit complicated and heavy though! /* PLUG: http://plug.org, #utah on irc.freenode.net Unsubscribe: http://plug.org/mailman/options/plug Don't fear the penguin. */
