[
https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262973#comment-15262973
]
Li Lu commented on MAPREDUCE-6424:
----------------------------------
Hi [~Naganarasimha], thanks for the work! I took a look at the patch. Most part
of the patch LGTM. One general concern: given the fact that we're returning
null for most of the HistoryEvents, maybe there's a better way to do this? My
proposal would be generalize the toTimelineEvent method into something like
addDataToEntity, so that we can attach any kinds of timeline v2 data to
entities (info, config, metric, event, etc,.) in different kinds of events. Due
to the nature of those HistoryEvents, it's quite possible that each event may
only touch a part of a timeline entity, so maybe we don't want to generate
events and/or metrics separately? I'm raising this point also because the
MapReduce code may act as an example for how to use the v2 APIs in general, so
we may want to figure out a "scalable" way to handle all cases. I'm terribly
sorry not getting back on this point earlier. Do we still have time to find a
solution here given the time is quite limited to our 1st milestone? Thanks!
nits:
JobHistoryEventHandler
l.1053 style, parameter timestamp not used?
l.1223 I agree with Sangjin's comments that we may probably want to use a
blocking call to post final data. On the performance side, however, is it
possible to post all contents in one batch, instead of making two calls?
> Store MR counters as timeline metrics instead of event
> ------------------------------------------------------
>
> Key: MAPREDUCE-6424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch,
> MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch
>
>
> In MAPREDUCE-6327, we make map/reduce counters get encoded from
> JobFinishedEvent as timeline events with counters details in JSON format.
> We need to store framework specific counters as metrics in timeline service
> to support query, aggregation, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
