Dmitry Lysnichenko created AMBARI-14764:
-------------------------------------------
Summary: MapReduce, Oozie service check failures after upgrade
finished
Key: AMBARI-14764
URL: https://issues.apache.org/jira/browse/AMBARI-14764
Project: Ambari
Issue Type: Bug
Reporter: Dmitry Lysnichenko
Assignee: Dmitry Lysnichenko
# MapReduce jobs were failing after upgrading and restarting MapReduce2 service
(ClassNotFound exception). Also Oozie job were failing because of the same
reason.
# This was caused because mapreduce.application.classpath property had the
older version of the LZO codec
(/usr/hdp/current/hadoop/lib/hadoop-lzo-0.6.0.<OLD-STACK-VERSION>.jar).
# During upgrade the path was valid because we had not switched
/usr/hdp/current/hadoop symlink to new version. After upgrade, once the symlink
is updated to new stack version and after restarting MapReduce2 this path
becomes invalid.
# To fix this updated the LZO codec path to
/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar in
mapreduce.application.classpath config property. Also in oozie-env property
added export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS”
Fix:
# During stack upgrade, update lzo codec path in
mapreduce.application.classpath to
{code}
/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar
{code}
Also, looks like we also need to update following property for TEZ.
{code}
<property>
<name>tez.cluster.additional.classpath.prefix</name>
<value>/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf/secure</value>
<description></description>
</property>
{code}
# Update oozie-env and add
{code}
export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS"
{code}
# Add upgrade pre-checks to validate if any config properties have
old-stack-version hardcodings and report these config properties to be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
