[ http://issues.apache.org/jira/browse/NUTCH-207?page=comments#action_12365462 ]
Rod Taylor commented on NUTCH-207: ---------------------------------- Code was by Radu Mateescu with additional kibitzing by myself. > Bandwidth target for fetcher rather than a thread count > ------------------------------------------------------- > > Key: NUTCH-207 > URL: http://issues.apache.org/jira/browse/NUTCH-207 > Project: Nutch > Type: New Feature > Components: fetcher > Versions: 0.8-dev > Reporter: Rod Taylor > Attachments: ratelimit.patch > > Increases or decreases the number of threads from the starting value > (fetcher.threads.fetch) up to a maximum (fetcher.threads.maximum) to achieve > a target bandwidth (fetcher.threads.bandwidth). > It seems to be able to keep within 10% of the target bandwidth even when > large numbers of errors are found or when a number of large pages is run > across. > To achieve more accurate tracking Nutch should keep track of protocol > overhead as well as the volume of pages downloaded. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
