[
https://issues.apache.org/jira/browse/MAPREDUCE-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224903#comment-14224903
]
Junping Du commented on MAPREDUCE-6168:
---------------------------------------
Thanks for comments [~kasha]!
bq. I would call the issue fixed in 2.x only if 2.2 client is able to talk to
2.x server and viceversa. This doesn't seem to be the case for 2.6. Let me know
if I am missing something here.
If 2.2 client cannot talk with 2.4 and 2.5 in the sever side, does the meaning
too much for 2.2 client to talk with 2.6 server? The problem exists in client
side that exception get thrown when getting new counters from server in new
version. If we want to fix it in server side, we should have some mechanism to
filter the new counter in server side with detecting the version of client. So
far from what I know, we don't have versioning differentiation for MR client
that make the fix in server side harder. If we are adding some new version
detection mechanism here after 2.6.x, then we cannot differentiate previous
release versions so Counter filter in server side still cannot work. Thoughts?
> Old MR client is still broken when receiving new counters from MR job
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6168
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6168
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Zhijie Shen
> Assignee: Junping Du
> Priority: Blocker
>
> In the following scenarios:
> 1. Either insecure or secure;
> 2. MR 2.2 with new shuffle on NM;
> 3. Submitting via old client.
> We will see the following console exception:
> {code}
> 14/11/17 14:56:19 INFO mapreduce.Job: Job job_1416264695865_0003 completed
> successfully
> java.lang.IllegalArgumentException: No enum constant
> org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_REDUCES
> at java.lang.Enum.valueOf(Enum.java:236)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> at
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
> at
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
> at
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
> at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
> at
> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {code}
> The problem is supposed to be fixed by MAPREDUCE-5831, however, it seems that
> we haven't cover all the problematic code path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
