[
https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264818#comment-15264818
]
Li Lu commented on MAPREDUCE-6424:
----------------------------------
bq. So thought better to postpone this decision to Varun Saxena's patch.
Thoughts?
Sure. Once we plan to refactor it I'm fine.
bq. TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling is
passing locally
I can reproduce the failure on my local machine:
{code}
Running org.apache.hadoop.mapred.TestMRTimelineEventHandling
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 56.764 sec <<<
FAILURE! - in org.apache.hadoop.mapred.TestMRTimelineEventHandling
testMRNewTimelineServiceEventHandling(org.apache.hadoop.mapred.TestMRTimelineEventHandling)
Time elapsed: 56.57 sec <<< FAILURE!
java.lang.AssertionError: Job Metrics are not published
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.mapred.TestMRTimelineEventHandling.verifyMetricsWhenEvent(TestMRTimelineEventHandling.java:361)
at
org.apache.hadoop.mapred.TestMRTimelineEventHandling.checkNewTimelineEvent(TestMRTimelineEventHandling.java:292)
at
org.apache.hadoop.mapred.TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling(TestMRTimelineEventHandling.java:210)
Results :
Failed tests:
TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling:210->checkNewTimelineEvent:292->verifyMetricsWhenEvent:361
Job Metrics are not published
{code}
>From the log not sure if it's related to some aggregation operations. I'm
>looking into it.
bq. we need to decide if we want to split configs right now and if yes is an
approach based on size configuration or a internal fixed value fine ?
Let's make the sizes configurable? Adding a config should be quick and then we
can decide on their names?
> Store MR counters as timeline metrics instead of event
> ------------------------------------------------------
>
> Key: MAPREDUCE-6424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Junping Du
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch,
> MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch,
> MAPREDUCE-6424-YARN-2928.v1.004.patch
>
>
> In MAPREDUCE-6327, we make map/reduce counters get encoded from
> JobFinishedEvent as timeline events with counters details in JSON format.
> We need to store framework specific counters as metrics in timeline service
> to support query, aggregation, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
