[ http://issues.apache.org/jira/browse/NUTCH-207?page=comments#action_12365462 ]
Rod Taylor commented on NUTCH-207: ---------------------------------- Code was by Radu Mateescu with additional kibitzing by myself. > Bandwidth target for fetcher rather than a thread count > ------------------------------------------------------- > > Key: NUTCH-207 > URL: http://issues.apache.org/jira/browse/NUTCH-207 > Project: Nutch > Type: New Feature > Components: fetcher > Versions: 0.8-dev > Reporter: Rod Taylor > Attachments: ratelimit.patch > > Increases or decreases the number of threads from the starting value > (fetcher.threads.fetch) up to a maximum (fetcher.threads.maximum) to achieve > a target bandwidth (fetcher.threads.bandwidth). > It seems to be able to keep within 10% of the target bandwidth even when > large numbers of errors are found or when a number of large pages is run > across. > To achieve more accurate tracking Nutch should keep track of protocol > overhead as well as the volume of pages downloaded. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
