[
https://issues.apache.org/jira/browse/MAPREDUCE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148477#comment-14148477
]
Junping Du commented on MAPREDUCE-5831:
---------------------------------------
bq. It's fine too. If so, we need to fix FileSystemCounterGroup.findCounter,
which will have the similar issue.
Good point. Address this in v2 version.
> Old MR client is not compatible with new MR application
> -------------------------------------------------------
>
> Key: MAPREDUCE-5831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5831
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mr-am
> Affects Versions: 2.2.0, 2.3.0
> Reporter: Zhijie Shen
> Assignee: Junping Du
> Priority: Blocker
> Attachments: MAPREDUCE-5831-v2.patch, MAPREDUCE-5831.patch
>
>
> Recently, we saw the following scenario:
> 1. The user setup a cluster of hadoop 2.3., which contains YARN 2.3 and MR
> 2.3.
> 2. The user client on a machine that MR 2.2 is installed and in the classpath.
> Then, when the user submitted a simple wordcount job, he saw the following
> message:
> {code}
> 16:00:41,027 INFO main mapreduce.Job:1345 - map 100% reduce 100%
> 16:00:41,036 INFO main mapreduce.Job:1356 - Job job_1396468045458_0006
> completed successfully
> 16:02:20,535 WARN main mapreduce.JobRunner:212 - Cannot start job
> [wordcountJob]
> java.lang.IllegalArgumentException: No enum constant
> org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_REDUCES
> at java.lang.Enum.valueOf(Enum.java:236)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> at
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
> at
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
> at
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
> at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
> . . .
> {code}
> The problem is that the wordcount job was running on one or more than one
> nodes of the YARN cluster, where MR 2.3 libs were installed, and
> JobCounter.MB_MILLIS_REDUCES is available in the counters. On the other side,
> due to the classpath setting, the client was likely to run with MR 2.2 libs.
> After the client retrieved the counters from MR AM, it tried to construct the
> Counter object with the received counter name. Unfortunately, the enum didn't
> exist in the client's classpath. Therefore, "No enum constant" exception is
> thrown here.
> JobCounter.MB_MILLIS_REDUCES is brought to MR2 via MAPREDUCE-5464 since
> Hadoop 2.3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
