On Thu, Apr 5, 2012 at 3:41 PM, Gabor Szabo <ga...@szabgab.com> wrote: > Hi, > > I am making progress with the crawler using > POE::Component::Client::HTTP but I encountered an issue. > Once in a while a requests fails with 408 Request Timeout and then - > it seems - all the subsequent requests fail. > Even the ones I post *after* receiving that timeout error. > > Is there a way tell the module to shut down a specific connection - > that might be broken - and open a new one? > > Any other idea how could I track down the problem? > > > Gabor
Just to reply to myself: Apparently the problem was that the default number of concurrent connections enabled by POE::Component::Client::Keepalive was lower than the number of requests I sent and the timeout period includes the time the requests waits in the queue as well. Creating a new POE::Component::Client::Keepalive with a high enough number for the max_per_host field, and assigning it to ConnectionManager fixed the problem. Or so I hope :) regards Gabor
