Lewis John McGibbney created NUTCH-3150:
-------------------------------------------
Summary: Expand Caching Hadoop Counter References
Key: NUTCH-3150
URL: https://issues.apache.org/jira/browse/NUTCH-3150
Project: Nutch
Issue Type: Sub-task
Components: metrics
Reporter: Lewis John McGibbney
Assignee: Lewis John McGibbney
Fix For: 1.22
This issue expands on initial work to cache counters in hot paths only. See
NUTCH-3141.
After some analysis this issue can be scoped as follows
9 files to modify (~45 counters total):
* Fetcher.java - 5 counters (main fetch loop)
* ResolverThread.java - 10 counters (DNS resolution)
* Generator.java - 9 counters (URL generation)
* CrawlDbReducer.java - Dynamic counters with Map
* Injector.java - 6 counters (mapper + reducer)
* CrawlDbFilter.java - 3 counters
* WebGraph.java - 2 counters
* DeduplicationJob.java - 1 counter
* CleaningJob.java - 1 counter
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
