[
https://issues.apache.org/jira/browse/NUTCH-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17903058#comment-17903058
]
ASF GitHub Bot commented on NUTCH-3083:
---------------------------------------
sebastian-nagel merged PR #834:
URL: https://github.com/apache/nutch/pull/834
> Add RobotRulesParser to bin/nutch
> ---------------------------------
>
> Key: NUTCH-3083
> URL: https://issues.apache.org/jira/browse/NUTCH-3083
> Project: Nutch
> Issue Type: Improvement
> Components: bin
> Affects Versions: 1.21
> Reporter: Sebastian Nagel
> Priority: Minor
> Fix For: 1.21
>
>
> The main method of the class {{org.apache.nutch.protocol.RobotRulesParser}}
> is quite useful if it's about verifying whether and how robots.txt files are
> parsed. It should be added to bin/nutch as *robotsparser*, similar to
> "parsechecker", "filterchecker", etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
