[
https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259240#comment-15259240
]
Sangjin Lee commented on MAPREDUCE-6424:
----------------------------------------
Thanks [~Naganarasimha] for your patch!
I think we said it is OK to emit them as is and capture the work of emitting
them periodically in a later JIRA.
One thing to add, though: I mentioned it elsewhere but we also need to emit the
metrics at the *YARN application* level if we want them aggregated to the
flows. Therefore, we should add code that emits the metrics for the YARN
application entity in addition to the MR job entity. I'd say that's actually
even more important than metrics at the job entity. The metrics at the
MR-specific entities are there to view, but they will not be aggregated. On the
other hand, the YARN application entity is the focal point of real time flow
aggregation.
> Store MR counters as timeline metrics instead of event
> ------------------------------------------------------
>
> Key: MAPREDUCE-6424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch
>
>
> In MAPREDUCE-6327, we make map/reduce counters get encoded from
> JobFinishedEvent as timeline events with counters details in JSON format.
> We need to store framework specific counters as metrics in timeline service
> to support query, aggregation, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
