OutlinkDB to emit unique URL's only
-----------------------------------
Key: NUTCH-1335
URL: https://issues.apache.org/jira/browse/NUTCH-1335
Project: Nutch
Issue Type: Bug
Affects Versions: 1.5
Reporter: Markus Jelsma
Assignee: Markus Jelsma
Fix For: 1.6
The OutlinkDB may contain duplicates if a segment is added more than once. The
aggregating code in the reducer is does not take care of removing duplicates.
See:
http://mail-archives.apache.org/mod_mbox/nutch-user/201204.mbox/%[email protected]%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
