Separating hadoop metrics log types at source for improving primary key lookup
------------------------------------------------------------------------------
Key: CHUKWA-497
URL: https://issues.apache.org/jira/browse/CHUKWA-497
Project: Chukwa
Issue Type: Improvement
Components: data collection
Affects Versions: 0.4.0, 0.3.0, 0.1.2
Environment: Java 6, RHEL 5.x
Reporter: Eric Yang
Chukwa Hadoop metrics log4j appender is using HadoopLog as a generic type to
capture all metrics from various metrics sources. i.e. JobTracker, TaskTracker,
Shuffle, Map, Reduce, NameNode, and DataNode, etc. It is best to make
separation between those types, hence each data type can have it's own properly
assigned key in demux phase by annotation.
- Shuffle, Map, Reduce, have key associated with task attempts or Job.
- JobTracker or TaskTracker have key associated with a host.
- NameNode or DataNode have key associated with a host.
HadoopMetricsProcessor should be deprecated in favor of separated demux classes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
