Jonathan Hurley created AMBARI-22245:
----------------------------------------
Summary: YARN Service Checks Fails Because of Old hadoop-client
Classpath Entry
Key: AMBARI-22245
URL: https://issues.apache.org/jira/browse/AMBARI-22245
Project: Ambari
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Jonathan Hurley
Assignee: Jonathan Hurley
Priority: Blocker
Fix For: 2.6.0
STR:
- Install basic YARN on HDP 2.6.0.0 (no Tez)
- Upgrade just YARN to HDP 2.6.0.2
Service checks will fail with the following:
{code}
17/10/12 17:55:20 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoSuchMethodError:
org.apache.hadoop.io.retry.RetryPolicies.retryForeverWithFixedSleep(JLjava/util/concurrent/TimeUnit;)Lorg/apache/hadoop/io/retry/RetryPolicy;
at
org.apache.hadoop.yarn.client.RMProxy.createRetryPolicy(RMProxy.java:280)
{code}
- All YARN daemons and clients are reporting 2.6.0.2
- All YARN daemons have loaded JARs for 2.6.0.2
What is happening here is that the applications being run on YARN are picking
up the older hadoop-common JAR file. The method {{retryForeverWithFixedSleep}}
did not exist in HDP 2.6.0.0.
It is picking up the older JARs for running applications because of the
{{yarn-site.xml}} property:
{code}
yarn-site.xml- <name>yarn.application.classpath</name>
yarn-site.xml:
<value>/etc/hadoop/conf,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*,/usr/hdp/current/ext/hadoop/*</value>
{code}
Ambari should be parameterizing the {{/usr/hdp/current/hadoop-client}} paths
here.
Whether or not this is an actual bug is debatable. We know that hadoop-client
would not be moving along with YARN and we were guaranteed that hadoop-client
would be 100% backward compatible. In this case, it's not because I was using a
pre-release version of 2.6.0.0. The same test from 2.6.0.2 to 2.6.0.3 succeeds.
We were also told that if hadoop-client needed a change, then it's basically a
full upgrade anyway.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
