[Nutch-dev] [jira] Commented: (NUTCH-446) RobotRulesParser should ignore Crawl-delay values of other bots in robots.txt

2007-05-10 Thread JIRA
[ https://issues.apache.org/jira/browse/NUTCH-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494734 ] Doğacan Güney commented on NUTCH-446: - So, does anyone have objections to this? It fixes an annoying (albeit

[Nutch-dev] [jira] Commented: (NUTCH-446) RobotRulesParser should ignore Crawl-delay values of other bots in robots.txt

2007-05-01 Thread Sami Siren (JIRA)
[ https://issues.apache.org/jira/browse/NUTCH-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492850 ] Sami Siren commented on NUTCH-446: -- +1 RobotRulesParser should ignore Crawl-delay values of other bots in