11:28 AM
To: nutch-user@lucene.apache.org
Subject: Re: Fetcher2 Slow
hi Roger :
you use new fecther with 1.0 or fetcher2 with 0.9 ? whicth is very slow?
and the problem is solved?
thanks!
--
View this message in context:
http://www.nabble.com/Fetcher2-Slow-tp21803185p23398585.html
Sent from
Not sure if that problem is solved, I have it and reported it in a previous
thread. Extremely fast fetch at the beginning and damn slow fetches after a
while.
Today I started a recrawl using the Wiki script I get a steady fetch
speed for about 1 hour so looks good. The only difference maybe
hi Roger :
you use new fecther with 1.0 or fetcher2 with 0.9 ? whicth is very slow?
and the problem is solved?
thanks!
--
View this message in context:
http://www.nabble.com/Fetcher2-Slow-tp21803185p23398585.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Roger Dunk wrote:
Andrzej stated in NUTCH-669 that some people reported performance
issues with Fetcher2, i.e. that it doesn't use the available bandwidth.
These reports are unconfirmed, and they may have been caused by
suboptimal URL / host distribution in a fetchlist - but it would be good
Hi Sami,
The machine has direct connectivity -- no NAT, and is not running IPV6.
Cheers...
Roger
--
From: Sami Siren ssi...@gmail.com
Sent: Monday, March 30, 2009 5:42 PM
To: nutch-user@lucene.apache.org
Subject: Re: Fetcher2 Slow
Roger Dunk
--
From: Roger Dunk ro...@at.com.au
Sent: Thursday, February 05, 2009 2:16 PM
To: nutch-user@lucene.apache.org
Subject: Re: Fetcher2 Slow
It makes no difference if I set fetcher.threads.per.host to 1 or 100,
which I assume is what you were
,
Roger
--
From: Roger Dunk ro...@at.com.au
Sent: Thursday, February 05, 2009 2:16 PM
To: nutch-user@lucene.apache.org
Subject: Re: Fetcher2 Slow
It makes no difference if I set fetcher.threads.per.host to 1 or 100,
which I assume is what you were
Roger Dunk wrote:
Andrzej stated in NUTCH-669 that some people reported performance
issues with Fetcher2, i.e. that it doesn't use the available bandwidth.
These reports are unconfirmed, and they may have been caused by
suboptimal URL / host distribution in a fetchlist - but it would be good
On Tue, Feb 3, 2009 at 4:10 AM, Roger Dunk ro...@at.com.au wrote:
Hi all,
I'm having no luck whatsoever using Fetcher2, as even with 50 threads enabled
and parsing disabled, I have 48 or 49 threads SpinWaiting, and 0 hosts in the
queue. I do however have some 50,000 pages to fetch, the
