[
https://issues.apache.org/jira/browse/MAPREDUCE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039704#comment-14039704
]
Zhijie Shen commented on MAPREDUCE-5831:
----------------------------------------
bq. This came up in the context of 2.4.1
bq. We solve this on our clusters by leveraging MAPREDUCE-442
and on mailing list:
bq. Actually, that is a reasonable expectation - particularly because we
should all be migrating towards MAPREDUCE-4421 and should stop installing MR on
every node.
Move it out of 2.4.1. Upon the discussion, do you think we even don't want to
fix the issue given we are supposed to deploy MR via distributed cache?
> Old MR client is not compatible with new MR application
> -------------------------------------------------------
>
> Key: MAPREDUCE-5831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5831
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mr-am
> Affects Versions: 2.2.0, 2.3.0
> Reporter: Zhijie Shen
> Assignee: Tan, Wangda
> Priority: Critical
>
> Recently, we saw the following scenario:
> 1. The user setup a cluster of hadoop 2.3., which contains YARN 2.3 and MR
> 2.3.
> 2. The user client on a machine that MR 2.2 is installed and in the classpath.
> Then, when the user submitted a simple wordcount job, he saw the following
> message:
> {code}
> 16:00:41,027 INFO main mapreduce.Job:1345 - map 100% reduce 100%
> 16:00:41,036 INFO main mapreduce.Job:1356 - Job job_1396468045458_0006
> completed successfully
> 16:02:20,535 WARN main mapreduce.JobRunner:212 - Cannot start job
> [wordcountJob]
> java.lang.IllegalArgumentException: No enum constant
> org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_REDUCES
> at java.lang.Enum.valueOf(Enum.java:236)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> at
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
> at
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
> at
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
> at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
> . . .
> {code}
> The problem is that the wordcount job was running on one or more than one
> nodes of the YARN cluster, where MR 2.3 libs were installed, and
> JobCounter.MB_MILLIS_REDUCES is available in the counters. On the other side,
> due to the classpath setting, the client was likely to run with MR 2.2 libs.
> After the client retrieved the counters from MR AM, it tried to construct the
> Counter object with the received counter name. Unfortunately, the enum didn't
> exist in the client's classpath. Therefore, "No enum constant" exception is
> thrown here.
> JobCounter.MB_MILLIS_REDUCES is brought to MR2 via MAPREDUCE-5464 since
> Hadoop 2.3.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
