[
https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259427#comment-15259427
]
Naganarasimha G R commented on MAPREDUCE-6424:
----------------------------------------------
Thanks for the comments [~sjlee0],
bq. I think we said it is OK to emit them as is and capture the work of
emitting them periodically in a later JIRA
Ok basically was trying to understand for the next jira, may be we can discuss
more there. Query was : Apart from metrics getting published at the end of each
task(M/R) we need to publish metrics when task(M/R) is *running* or have some
way to share running task's (M/R) metrics to AM(via umbilical proto) and AM
publish aggregated running tasks (M/R) metrics periodically ?
bq. but we also need emit the metrics at the YARN application level if we want
them aggregated to the flows. Therefore, we should add code that emits the
metrics for the YARN application entity in addition to the MR job entity.
This seems to be a good idea for oozie like flows but just one query, would it
be good for app to publish metrics in SystemEntities ? IIRC YARN-3401 was
raised to block it later on ? so do we need to support some other mechanism to
publish only metrics ?
> Store MR counters as timeline metrics instead of event
> ------------------------------------------------------
>
> Key: MAPREDUCE-6424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch
>
>
> In MAPREDUCE-6327, we make map/reduce counters get encoded from
> JobFinishedEvent as timeline events with counters details in JSON format.
> We need to store framework specific counters as metrics in timeline service
> to support query, aggregation, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
