[
https://issues.apache.org/jira/browse/MAPREDUCE-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916719#action_12916719
]
Arun C Murthy commented on MAPREDUCE-901:
-----------------------------------------
bq. I'm proposing a new approach to this issue: refactor the counter framework
(while keeping the existing api) so that it's possible to have different
implementation of counter groups, one of which is the FrameworkCounterGroup
(see attached file.)
This seems reasonable... I would be interested in a more sketched out version
of the patch. Btw, can you ensure (unit tests) that end-user code for
JobSubmissionProtocol.getCounters has no regression? Thanks.
> Move Framework Counters into a TaskMetric structure
> ---------------------------------------------------
>
> Key: MAPREDUCE-901
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-901
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 0.21.0
> Reporter: Owen O'Malley
> Assignee: Luke Lu
> Attachments: 901_1.patch, 901_1.patch, FrameworkCounterGroup.java,
> MAPREDUCE-901.patch, MAPREDUCE-901.patch
>
>
> I think we should move all of the Counters that the framework updates into a
> single class called TaskMetrics. TaskMetrics would have specific fields for
> each of the metrics like input records, input bytes, output records, etc.
> It would both reduce the serialized size of the heartbeats (by shrinking the
> Counters down to just the user's counters) and decrease the latency for
> updates to the JobTracker (since Counters are sent at most 1/minute instead
> of 1/heartbeat).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
