[
https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265023#comment-15265023
]
Li Lu commented on MAPREDUCE-6424:
----------------------------------
Hmm I think I found the root cause of the UT failure on
testMRNewTimelineServiceEventHandling. Note that in the unit test we're
directly reading the entity file from the file system. This is different to the
implementation of file system based timeline reader, where we merge all updates
to the same entity before we return. With the newly added aggregation logic,
there are aggregation updates to the application entity. Therefore, the
processing logic in verifyMetricsWhenEvent needs to handle this kind of
entities (right now it's an empty update). Maybe we'd like to use the file
system timeline reader to do this, since there we have a more comprehensive
reader implementation? Right now the logic in verifyMetricsWhenEvent seems to
be a little bit fragile to future changes?
There are certainly improvements we can do on the aggregation side. Let's
address that after the first milestone.
> Store MR counters as timeline metrics instead of event
> ------------------------------------------------------
>
> Key: MAPREDUCE-6424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch,
> MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch,
> MAPREDUCE-6424-YARN-2928.v1.004.patch
>
>
> In MAPREDUCE-6327, we make map/reduce counters get encoded from
> JobFinishedEvent as timeline events with counters details in JSON format.
> We need to store framework specific counters as metrics in timeline service
> to support query, aggregation, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
