Are you expecting the cookie to be resent within the same crawl cycle or in an another crawl cycle? By 'crawl cycle' I mean one invocation of fetcher in a particular depth.
If you have multiple crawl cycles (i.e. depth > 1), then the cookie which was received in the crawl for one depth can not be used in a crawl at another depth. At every depth, the fetcher is invoked and it dies when the cycle is over. So, the cookies do not persist between fetches of different depths. Regards, Susam Pal On Thu, Dec 11, 2008 at 3:29 PM, George Herlin <[EMAIL PROTECTED]> wrote: > I have read that if one sets the plugin.includes property to use > protocol-httpclient, Nutch 0.9 will use the Apache Commons httpclient. > > I also believe (with some reason) that that httpclient does automatic cookie > management. > > My logs tell me that I am configured to use said protocol-httpclient: > ... > 2008-12-11 13:47:22,752 INFO plugin.PluginRepository - Http / Https > Protocol Plug-in (protocol-httpclient) > ... > > Yet, no cookies seem to be resent to server. > > Is there any way in which I can verify what Set-Cookies have been stored? Or > is there some extra bit of configuration to enable this stuff? > > Thanks in advance >
