Sebastian Nagel created NUTCH-3083:
--------------------------------------
Summary: Add RobotRulesParser to bin/nutch
Key: NUTCH-3083
URL: https://issues.apache.org/jira/browse/NUTCH-3083
Project: Nutch
Issue Type: Improvement
Components: bin
Affects Versions: 1.21
Reporter: Sebastian Nagel
Fix For: 1.21
The main method of the class {{org.apache.nutch.protocol.RobotRulesParser}} is
quite useful if it's about verifying whether and how robots.txt files are
parsed. It should be added to bin/nutch as *robotschecker*, similar to
"parsechecker", "filterchecker", etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
