[
https://issues.apache.org/jira/browse/AMBARI-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570237#comment-14570237
]
Hudson commented on AMBARI-11630:
---------------------------------
SUCCESS: Integrated in Ambari-trunk-Commit #2792 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/2792/])
AMBARI-11630 - Falcon Server Seems To Have Hadoop 2.2 Paths After Upgrade
(jonathanhurley) (jhurley:
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=b7e1bcebf8244a63ffc594928dbb278370261c68)
* ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py
*
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
*
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
*
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
> Falcon Server Seems To Have Hadoop 2.2 Paths After Upgrade
> ----------------------------------------------------------
>
> Key: AMBARI-11630
> URL: https://issues.apache.org/jira/browse/AMBARI-11630
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.1.0
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Critical
> Fix For: 2.1.0
>
> Attachments: AMBARI-11630.patch
>
>
> After an upgrade from HDp 2.2 to HDP 2.3, Falcon seems to have a mixture of
> HDP 2.2 and 2.3 paths in its process path:
> {noformat}
> [root@jhurley-hdp22-ru-6 ~]# ps aux | grep falcon
> falcon 16231 0.2 6.0 3623356 455580 ? Sl 14:34 0:18
> /usr/jdk64/jdk1.8.0_40/bin/java -Xmx1024m -noverify -Dfalcon.embeddedmq=True
> -Dfalcon.emeddedmq.port=61616 -Dfalcon.log.dir=/var/log/falcon
> -Dfalcon.embeddedmq.data=/hadoop/falcon/embeddedmq/data
> -Dfalcon.home=/usr/hdp/current/falcon-server
> -Dconfig.location=/usr/hdp/current/falcon-server/conf
> -Dfalcon.app.type=falcon -Dfalcon.catalog.service.enabled= -cp
> /usr/hdp/current/falcon-server/conf:/usr/hdp/2.2.7.0-2808/hadoop/conf:/usr/hdp/2.2.7.0-2808/hadoop/lib/*:/usr/hdp/2.2.7.0-2808/hadoop/.//*:/usr/hdp/2.2.7.0-2808/hadoop-hdfs/./:/usr/hdp/2.2.7.0-2808/hadoop-hdfs/lib/*:/usr/hdp/2.2.7.0-2808/hadoop-hdfs/.//*:/usr/hdp/2.2.7.0-2808/hadoop-yarn/lib/*:/usr/hdp/2.2.7.0-2808/hadoop-yarn/.//*:/usr/hdp/2.2.7.0-2808/hadoop-mapreduce/lib/*:/usr/hdp/2.2.7.0-2808/hadoop-mapreduce/.//*::/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/tez-client/*:/usr/hdp/current/tez-client/lib/*:/etc/tez/conf/:/usr/hdp/2.2.7.0-2808/tez/*:/usr/hdp/2.2.7.0-2808/tez/lib/*:/etc/tez/conf:/usr/hdp/current/falcon-server/server/webapp/falcon/WEB-INF/classes:/usr/hdp/current/falcon-server/server/webapp/falcon/WEB-INF/lib/*:/usr/hdp/current/falcon-server/libext/*
> org.apache.falcon.Main -app
> /usr/hdp/current/falcon-server/server/webapp/falcon -port 15000
> root 28434 0.0 0.0 103252 860 pts/5 S+ 16:44 0:00 grep falcon
> [root@jhurley-hdp22-ru-6 ~]# ll /usr/hdp/current | grep falcon
> lrwxrwxrwx. 1 root root 28 May 29 03:30 falcon-client ->
> /usr/hdp/2.2.7.0-2808/falcon
> lrwxrwxrwx. 1 root root 28 May 29 14:34 falcon-server ->
> /usr/hdp/2.3.0.0-2162/falcon
> {noformat}
> A restart of Falcon post-upgrade corrects these problems, so it seems like
> it's an issue only during upgrade.
> Also, {{falcon-client}} is not upgraded as part of the upgrade process. This
> may be covered by the {{hdp-select all}} work capture in another Jira, but
> falcon-server may still need the client updated ahead of that.
> The root cause of this is the fact that Ambari is not setting {{HADOOP_HOME}}
> when invoking Falcon. Under most circumstances, this is OK since Falcon's
> {{falcon-config.sh}} script will then try to figure it out with {{which
> hadoop}}:
> {code:title=falcon-config.sh HDP 2.2}
> HADOOPDIR=`which hadoop`
> if [ "$HADOOPDIR" != "" ]; then
> echo "Hadoop is installed, adding hadoop classpath to falcon classpath"
> FALCONCPPATH="${FALCONCPPATH}:`hadoop classpath`"
> elif [ "$HADOOP_HOME" != "" ]; then
> echo "Hadoop home is set, adding libraries from
> '${HADOOP_HOME}/bin/hadoop classpath' into falcon classpath"
> FALCONCPPATH="${FALCONCPPATH}:`${HADOOP_HOME}/bin/hadoop classpath`"
> else
> echo "Could not find installed hadoop and HADOOP_HOME is not set."
> echo "Using the default jars bundled in ${BASEDIR}/hadooplibs/"
> FALCONCPPATH="${FALCONCPPATH}:${BASEDIR}/hadooplibs/*"
> fi
> {code}
> The problem in HDP 2.2 is that it will always find the {{HADOOPDIR}} first
> and never pickup {{HADOOP_HOME}}. In an upgraded environment, it will always
> use the 2.2 libs.
> This appears to heave been fixed in HDP 2.3:
> {code:title=falcon-config.sh HDP 2.3}
> FALCONCPPATH="$FALCON_CONF"
> HADOOPDIR=`which hadoop`
> if [ "$HADOOP_HOME" != "" ]; then
> echo "Hadoop home is set, adding libraries from
> '${HADOOP_HOME}/bin/hadoop classpath' into falcon classpath"
> FALCONCPPATH="${FALCONCPPATH}:`${HADOOP_HOME}/bin/hadoop classpath`"
> elif [ "$HADOOPDIR" != "" ]; then
> echo "Hadoop is installed, adding hadoop classpath to falcon classpath"
> FALCONCPPATH="${FALCONCPPATH}:`hadoop classpath`"
> else
> echo "Could not find installed hadoop and HADOOP_HOME is not set."
> echo "Using the default jars bundled in ${BASEDIR}/hadooplibs/"
> FALCONCPPATH="${FALCONCPPATH}:${BASEDIR}/hadooplibs/*"
> fi
> {code}
> So we just need to ensure we pass in {{HADOOP_HOME}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
