speed of fetcher in nutch-2.0

2012-08-23 Thread alxsss
Hello, I am using nutch-2.0 with hbase-0.92.1. I noticed that, in depth 1, 2,3 fetcher was fetching around 20K urls per hour. In depth 4 it fetches only 8K urls per hour. Any ideas what could cause this decrease in speed. I use local mode with 10 threads. Thanks. Alex.

Auto-Re: speed of fetcher in nutch-2.0

2012-08-23 Thread fu xiang hua
您的邮件已收到，谢谢！

Re: speed of fetcher in nutch-2.0

2012-08-23 Thread Walter Tietze
Am 23.08.2012 20:36, schrieb alx...@aim.com: Hello, I am using nutch-2.0 with hbase-0.92.1. I noticed that, in depth 1, 2,3 fetcher was fetching around 20K urls per hour. In depth 4 it fetches only 8K urls per hour. Any ideas what could cause this decrease in speed. I use local mode

Re: speed of fetcher in nutch-2.0

2012-08-23 Thread Lewis John Mcgibbney
Hi, @Alxsss I hope Walters suggestion(s) help you out here. @Walter I've added your model answer to the wiki [0] this is a great response and I just couldn't help but add it. Thank you Lewis [0]