hi,
qi wu wrote: > Hi, I am using Fetcher2 with 200 threads started. I get a satisify > speed(about 20pages/s) at the beginning stage ,but after no more > than one hour,there are many spinWaiting threads. Where might be the > bottleneck? network, memory or anyplace else? Could you also give me > some hints on how to get more detailed debug info? Not specific to fetcher2, but how are the pages distributed among different hosts in fetchlist? Have you configured reasonable setting for generate.max.per.host in nutch conf? If you generate too many pages for too few hosts there's no way fetcher|fetcher2 can fetch them fast unless you make it non polite. -- Sami Siren ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
