I have read that you don't want to make more than 1 or
2 requests per second to the same host, or else you
will start adversely affecting their bandwidth. Is
this a good rule of thumb?
Along those lines, what would be the best values to
put in the nutch config file to maximize speed of
fetching without hammering the site? I'm thinking
something like this:
fetcher.server.delay: 1.0
fetcher.threads.per.host: 2
thanks,
raymond
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
