Markus Jelsma created NUTCH-3096:
------------------------------------
Summary: HostDB ResolverThread can create too many job counters
Key: NUTCH-3096
URL: https://issues.apache.org/jira/browse/NUTCH-3096
Project: Nutch
Issue Type: Bug
Components: hostdb
Affects Versions: 1.20
Reporter: Markus Jelsma
Assignee: Markus Jelsma
Fix For: 1.21
Hadoop will allow no more than 120 distinct counters. If we have a large number
of distinct DNS lookup failure counts, we'll exceed the limit, Hadoop will
complain, the job will fail.
Let's limit the amount of possibilities by grouping the numFailures in just a
few buckets.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
