Markus Jelsma created NUTCH-3120:
------------------------------------
Summary: Automatically increase crawl-delay on HTTP 429
Key: NUTCH-3120
URL: https://issues.apache.org/jira/browse/NUTCH-3120
Project: Nutch
Issue Type: Improvement
Reporter: Markus Jelsma
Assignee: Markus Jelsma
Fix For: 1.22
Thought i remember a discussion or ticket on this subject, but it seems no code
of this sort is in master at the moment.
Anyway, small patch that adds HTTP429 to ProtocolStatus, the setter for that
status to HttpBase, and the reading of that status in FetcherThread, so it can
adjust the fetch speed (hardcoded to *3 right now).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
