I have read that you don't want to make more than 1 or
2 requests per second to the same host, or else you
will start adversely affecting their bandwidth. Is
this a good rule of thumb?
Along those lines, what would be the best values to
put in the nutch config file to maximize speed of
fetching without hammering the site? I'm thinking
something like this:
fetcher.server.delay: 1.0
fetcher.threads.per.host: 2
thanks,
raymond
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
