[ https://issues.apache.org/jira/browse/NUTCH-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15069421#comment-15069421 ]
Markus Jelsma commented on NUTCH-2065: -------------------------------------- Agreed! > Domain URL filter to support protocols > -------------------------------------- > > Key: NUTCH-2065 > URL: https://issues.apache.org/jira/browse/NUTCH-2065 > Project: Nutch > Issue Type: Improvement > Affects Versions: 1.10 > Reporter: Markus Jelsma > Attachments: NUTCH-2065.patch, NUTCH-2065.patch > > > The filter allows all protocols for all whitelisted domains, hosts or > suffixes but it usually makes little sense to index both http and https URL's > of the same domain. This is not unlike the host URL filter, which prevents > indexing of duplicate hosts e.g. apache.org and www.apache.org. -- This message was sent by Atlassian JIRA (v6.3.4#6332)