Hi Andrzej.
ok. make sense, thanks a lot for the clarification.
marko
On Oct 14, 2009, at 2:46 PM, Andrzej Bialecki wrote:
Marko Bauhardt wrote:
hi.
is there a way for using http-keep-alive with nutch?
supports protocol-http or protocol-httpclient keep alive?
i cant find the using of http-keep-alive inside the code or in
configuration files?
protocol-httpclient can support keep-alive. However, I think that it
won't help you much. Please consider that Fetcher needs to wait some
time between requests, and in the meantime it will issue requests to
other sites. This means that if you want to use keep-alive
connections then the number of open connections will climb up
quickly, depending on the number of unique sites on your fetchlist,
until you run out of available sockets. On the other hand, if the
number of unique sites is small, then most of the time the Fetcher
will wait anyway, so the benefit from keep-alives (for you as a
client) will be small - though there will be still some benefit for
the server side.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
