Hi,
I'm not sure if this the problem because the RequestScheduler hangs, but looks like that could be an reason.
java.lang.ArithmeticException: / by zero
at net.nutch.fetcher.RequestScheduler.run(RequestScheduler.java:1271)
at net.nutch.fetcher.RequestScheduler.main(RequestScheduler.java:1449)
That are this lines of code:
<x-tad-bigger> </x-tad-bigger><x-tad-bigger>// decide how many threads to throttle</x-tad-bigger><x-tad-bigger>
</x-tad-bigger><x-tad-bigger>int</x-tad-bigger><x-tad-bigger> kbitsPerThread= recentBandwidth
/ (numFetchThreads - curNumThreadsThrottled);
</x-tad-bigger><x-tad-bigger>int</x-tad-bigger><x-tad-bigger> newNumThreadsThrottled=
numFetchThreads - (throttleMaxBandwidth / kbitsPerThread);
So the problem is a classical rounding problem.
I suggest to handle kbitsPerThread as double.
</x-tad-bigger><x-tad-bigger>double</x-tad-bigger><x-tad-bigger> kbitsPerThread= recentBandwidth
/ (numFetchThreads - curNumThreadsThrottled);
</x-tad-bigger><x-tad-bigger>int</x-tad-bigger><x-tad-bigger> newNumThreadsThrottled= (</x-tad-bigger><x-tad-bigger>int</x-tad-bigger><x-tad-bigger>) Math.round(
numFetchThreads - (throttleMaxBandwidth / kbitsPerThread));
As reason of this change we should declared percentChangeInBandwidth and lastKbitsPerThread as double as well.
Sorry I can provide a patch since my RequestScheduler is out of cvs sync because of my last SNMP patch.
</x-tad-bigger>
I will make a test if this will fix the problems of a hanging requestScheduler and let you know.
HTH
Sefan
---------------------------------------------------------------
open technology: http://www.media-style.com
open source: http://www.weta-group.net
open discussion: http://www.text-mining.org
