Hi, See https://issues.apache.org/jira/browse/NUTCH-721 for an explanation. This has been fixed in the SVN version.
Julien -- DigitalPebble Ltd http://www.digitalpebble.com 2010/1/20 axi <axi...@gmail.com>: > > Hi to all, > I'm a novice user of Nutch, I have it on a debian machine, and I have probe > the latest release of nutch 1.0 with very slow results in crawling, I have a > 10 Megabytes/s connection and it only crawls at 300 Kb/s with peaks of 1 > Mb/s. I tweaked everything, dns, linux tcp settings, thread numbers, java > conf etc.. but anything have effect. There are a lot of spin waiting threads > there, only 10-20 of them working and I have injected 1M different hosts, so > politeness is not the problem. I swithched back to 0.9 nutch, and then it > works like a charm at good speeds 5-6 Mb/s with the bottleneck on machine > cpu. > > ¿Are this issues solved on dev version of nutch or why is this happens? > > Thanks in advance, > > -- > View this message in context: > http://old.nabble.com/Nutch-1.0-slow-crawls-tp27243302p27243302.html > Sent from the Nutch - User mailing list archive at Nabble.com. > >
