i don't know if they're really "waiting" but i'm almost sure you'll just get 5 threads fetching. if it's a problem for you i think u can turn this restriction off. have an eye on the comments in nutch-default. it includes a pair or restrictions for the same host like max_number_of request or similar. have a look also on that.
alex On Thu, Mar 26, 2009 at 4:01 PM, Alex Basa <alex_b...@yahoo.com> wrote: > > Can someone please explain to me > > fetcher.threads.fetch > > and > > fetcher.threads.per.host > > Does it mean that if I am only crawling one site and have > > fetcher.threads.per.host = 5 > > fetcher.threads.fetch = 100 > > that we will only have 5 threads fetching from the one host and the other > 95 threads will be waiting? > > I couldn't find any documentation on this. Thanks in advance, > > Alex > > > > >
