This is normal and means the fetcher is finishing all it's input URL's and
writing stuff to disk.
-----Original message-----
> From:pepe3059 <[email protected]>
> Sent: Sat 02-Jun-2012 22:15
> To: [email protected]
> Subject: threads disminution when fetching page
>
> Hello, i hope you can help me
>
>
> i am doing a crawling from a country, what i did was a script for avoid an
> interrupted process configured for 20 threads. But when a certain page is
> reached, the crawler behaves rarely stoping all threads (20) . Below is the
> log, no one message of fetcher done is at the end and the processor is at
> 98% of usage. I don't know what happens, the server's page is not blocking
> me (tested with wget) and i have enough disk space
>
>
> * queue: http://www.frente.com.mx
> maxThreads = 1
> inProgress = 0
> crawlDelay = 6000
> minCrawlDelay = 0
> nextFetchTime = 1338578382163
> now = 1338578381900
> 0.
> http://www.frente.com.mx/2012/03/01/rockabilly-radiografia-de-un-copete/
> fetching
> http://www.frente.com.mx/2012/03/01/rockabilly-radiografia-de-un-copete/
> -finishing thread FetcherThread, activeThreads=19
> -finishing thread FetcherThread, activeThreads=18
> -finishing thread FetcherThread, activeThreads=17
> -finishing thread FetcherThread, activeThreads=16
> -finishing thread FetcherThread, activeThreads=15
> -finishing thread FetcherThread, activeThreads=14
> -finishing thread FetcherThread, activeThreads=13
> -finishing thread FetcherThread, activeThreads=12
> -finishing thread FetcherThread, activeThreads=11
> -finishing thread FetcherThread, activeThreads=10
> -finishing thread FetcherThread, activeThreads=9
> -finishing thread FetcherThread, activeThreads=8
> -finishing thread FetcherThread, activeThreads=7
> -activeThreads=7, spinWaiting=6, fetchQueues.totalSize=0
> -finishing thread FetcherThread, activeThreads=6
> -finishing thread FetcherThread, activeThreads=5
> -finishing thread FetcherThread, activeThreads=4
> -finishing thread FetcherThread, activeThreads=3
> -finishing thread FetcherThread, activeThreads=2
> -finishing thread FetcherThread, activeThreads=1
> -activeThreads=1, spinWaiting=0, fetchQueues.totalSize=0
> -finishing thread FetcherThread, activeThreads=0
> -activeThreads=0, spinWaiting=0, fetchQueues.totalSize=0
> -activeThreads=0
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/threads-disminution-when-fetching-page-tp3987381.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
