Remove remaining robots/IP blocking code in lib-http
----------------------------------------------------
Key: NUTCH-876
URL: https://issues.apache.org/jira/browse/NUTCH-876
Project: Nutch
Issue Type: Bug
Components: fetcher
Affects Versions: 2.0
Reporter: Andrzej Bialecki
Assignee: Andrzej Bialecki
There are remains of the (very old) blocking code in
lib-http/.../HttpBase.java. This code was used with the OldFetcher to manage
politeness limits. New trunk doesn't have OldFetcher anymore, so this code is
useless. Furthermore, there is an actual bug here - FetcherJob forgets to set
Protocol.CHECK_BLOCKING and Protocol.CHECK_ROBOTS to false, and the defaults in
lib-http are set to true.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
