On Wed, Jul 05, 2006 at 07:58:29PM +0200, Roland Weber wrote: > > Okay, so I should keep cookies in my own code and pass them to executeMethod > > either with using HttpState object or with adding cookie directly with > > addcookie method. > > No. You should let HttpClient handle cookies in HttpState. But your code > needs to make sure that there is a separate HttpState for each session, or > each set of cookies you want to maintain. > > > Could you please explain why you are using HostConfiguration > > and why it needs to be created for each thread? > > The HttpClient method that allows for passing an HttpState also expects > a HostConfiguration: > http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpClient.html#executeMethod(org.apache.commons.httpclient.HostConfiguration,%20org.apache.commons.httpclient.HttpMethod,%20org.apache.commons.httpclient.HttpState) > > The HostConfiguration object could get updated during method execution, > in particular if the method director has to follow a redirect. That's > why you don't want different threads to share HttpConfiguration objects.
It seems I need to explain my situation, since I feel we are talking about different things. I had developed a spider application. This application has 1 main thread (processor), which maintains URL queue and set of processed URLs. This processor thread takes URLs from it's queue, creates consumer thread and passess url to consumer thread. Consumer thread takes an url, gets a page and parses it, extracts URLs which are pushed back to queue, and so on. So in general we're starting with 1 single URL, which could set some cookies needed by any other URL which will be processed later. I think I need to collect ALL cookies set by each page somewhere, because implementation of sophisticated cookie management like passign cookie only to children of current processed page will be some overkill for now. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
