Fetcher2 too many spinWaiting, How to tune?

[qi wu]

Hi,
  I am using  Fetcher2 with 200 threads started. I get a satisify speed(about 
20pages/s)  at the beginning stage ,but after no more than one hour,there are 
many spinWaiting threads. 
    "2007-04-02 15:32:07,350 WARN  fetcher.Fetcher2 - -activeThreads=199, 
spinWaiting=198, fetchQueues.totalSize=10000"
Below are the configurations:
Seed URLs:  2000,Only one URL for one website.
OS: Linux 2.4.21
CPU : Intel(R) Xeon(TM) CPU 3.00GHz *2
Memory: 2G

CPU usage is very low,no more than 50%.Memory usage is very high,only 10% 
memory left during fetch,start crawl with "-xmx1000M".The thread dump thread 
show they all are waiting at "Thread.sleep(500)".
      spinWaiting.incrementAndGet(500);
       try {
        Thread.sleep(500);
       } catch (Exception e) {
       }
       spinWaiting.decrementAndGet();
Where might be the bottleneck? network, memory or anyplace else? Could you also 
give me some hints on how to get more detailed debug info?

Thanks
-Qi