> What website are you working on?
Many different ones, each with their own nutch
configurations, which is why I'm trying to figure out
how to tweak the fetcher so it maximizes speed while
minimizing errors and webmaster annoyance. :)
Currently it seems to be working pretty well with just
using one thread like you suggested, and dropping the
fetcher.server.delay to a pretty low value, about 0.2
seconds. I'm averaging about 1-2 pages/sec on most
sites I've been testing it on, with few http errors.
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO September
19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
