Yoav, I think I replied to this one - set protocol-httpclient in nutch-site.xml
In short, it looks like there is no easy solution for this with Nutch as it is today. Once HostDB functionality is in place, this should be a lot easier to do. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Yoav Shapira <[EMAIL PROTECTED]> > To: [email protected] > Sent: Thursday, May 8, 2008 11:22:50 AM > Subject: Re: How to authenticate with cookies? > > On Thu, May 8, 2008 at 11:14 AM, Andrzej Bialecki wrote: > > * you have to use protocol-httpclient. There is no support for cookies in > > protocol-http. > > OK, how do I make sure protocol-httpclient is used? > > > * your fetchlist needs to have more than 1 url from the host - the first > > request will presumably set the cookies, if you are lucky. ;) > > No, the first fetch will ask for authentication. I want to get past > this point by supplying the cookies myself, that's why I asked the > question ;) > > Thanks, > > Yoav
