Markus Jelsma created NUTCH-2216:
------------------------------------
Summary: ignore.internal.links to optionally follow internal
redirects
Key: NUTCH-2216
URL: https://issues.apache.org/jira/browse/NUTCH-2216
Project: Nutch
Issue Type: Improvement
Components: fetcher
Affects Versions: 1.11
Reporter: Markus Jelsma
Assignee: Markus Jelsma
Priority: Minor
Fix For: 1.12
db.ignore.internal.links doesn't follow any internal hyperlinks or redirects.
Together with db.ignore.external.links it helps to restrict the crawl to a
predefined set of URL's, for example provided by a customer.
In many cases, a few of those URL's are redirects, which are not followed. This
issue adds an option to optionally allow internal redirects despite
db.ignore.internal.links being enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
