[
https://issues.apache.org/jira/browse/NUTCH-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Markus Jelsma updated NUTCH-1335:
---------------------------------
Description: The aggregating code in the Outlink reducer does not take care
of incoming duplicates. When the input segments contain duplicates of a single
URL they are collected. (was: The OutlinkDB may contain duplicates if a
segment is added more than once. The aggregating code in the reducer is does
not take care of removing duplicates.
See:
http://mail-archives.apache.org/mod_mbox/nutch-user/201204.mbox/%[email protected]%3E)
Patch Info: Patch Available
Summary: OutlinkDB to collect unique URL's only (was: OutlinkDB to
emit unique URL's only)
> OutlinkDB to collect unique URL's only
> --------------------------------------
>
> Key: NUTCH-1335
> URL: https://issues.apache.org/jira/browse/NUTCH-1335
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 1.5
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.6
>
>
> The aggregating code in the Outlink reducer does not take care of incoming
> duplicates. When the input segments contain duplicates of a single URL they
> are collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
