Re: Nutch use a Browser or phantomjs as fetcher

2014-06-21 Thread remi tassing
Hi, I'm planning on modifying protocol-httpclient (HttpResponse.java) based on this PhantonJSDriver tutorial: http://assertselenium.com/2013/03/25/getting-started-with-ghostdriver-phantomjs/ I will let you know how it works out Remi On Wed, Jun 11, 2014 at 5:25 AM, Julien Nioche

Re: Nutch use a Browser or phantomjs as fetcher

2014-06-10 Thread Julien Nioche
Hi You can do that as a custom protocol implementation. The fetcher code would stay the same but the byte content returned for a given URL would be produced by phantomjs or whichever selenuim backend you'd to use. HTH Julien On 7 June 2014 11:35, remi tassing tassingr...@gmail.com wrote:

Re: Nutch use a Browser or phantomjs as fetcher

2014-06-10 Thread Patrick Kirsch
Hey, On 06/10/2014 10:52 AM, Julien Nioche wrote: Hi You can do that as a custom protocol implementation. The fetcher code would stay the same but the byte content returned for a given URL would be produced by phantomjs or whichever selenuim backend you'd to use. Do you have a

Re: Nutch use a Browser or phantomjs as fetcher

2014-06-10 Thread Julien Nioche
Hi Patrick You could look at the protocol-http plugin as an example. Julien On 10 June 2014 10:22, Patrick Kirsch pkir...@zscho.de wrote: Hey, On 06/10/2014 10:52 AM, Julien Nioche wrote: Hi You can do that as a custom protocol implementation. The fetcher code would stay the same but

Nutch use a Browser or phantomjs as fetcher

2014-06-07 Thread Patrick Kirsch
Hey list, I'm sure this issue was asked several times, but a quick look in the nutch user archive did not help, so: Has anyone documentation or tried to use a browser (like chromium) or phantomjs etc. for fetching web pages? Due to a heavily loaded javascript site, nutch needs to see the

Re: Nutch use a Browser or phantomjs as fetcher

2014-06-07 Thread remi tassing
I'm currently looking at those separately but an integrated option would be more efficient. Looking forward for any experience sharing On Sat, Jun 7, 2014 at 6:25 PM, Patrick Kirsch pkir...@zscho.de wrote: Hey list, I'm sure this issue was asked several times, but a quick look in the nutch