balashashanka commented on a change in pull request #537:
URL: https://github.com/apache/nutch/pull/537#discussion_r452858141
##########
File path: src/java/org/apache/nutch/protocol/RobotRulesParser.java
##########
@@ -376,13 +379,18 @@ public int run(String[] args) {
*/
private static class TestRobotRulesParser extends RobotRulesParser {
- public TestRobotRulesParser(Configuration conf) {
+ public void setConf(Configuration conf) {
// make sure that agent name is set so that setConf() does not complain,
Review comment:
Hi @sebastian-nagel, a small question. Is this comment still valid,
since if the user-agent is not provided we will check with robots.agent and
others right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
