----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26591/#review56282 -----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml <https://reviews.apache.org/r/26591/#comment96580> Does "-d" check work for sym links? - Sumit Mohanty On Oct. 11, 2014, 3:23 a.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26591/ > ----------------------------------------------------------- > > (Updated Oct. 11, 2014, 3:23 a.m.) > > > Review request for Ambari, Andrew Onischuk, Dmytro Sen, Hitesh Shah, and > Sumit Mohanty. > > > Bugs: AMBARI-7739 > https://issues.apache.org/jira/browse/AMBARI-7739 > > > Repository: ambari > > > Description > ------- > > Create an Ambari 1.7.0 cluster with HDP 2.2.1, and install HDFS, Tez, MR, > Hive, Zookeeper. > > After switching the hive.execution.engine from "mr" to "tez", attempting to > run the hive shell generates an error. > > Logging initialized using configuration in > file:/etc/hive/conf.server/hive-log4j.properties > Exception in thread "main" java.lang.RuntimeException: > org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:670) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already > shutdown > at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:598) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:119) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454) > ... 8 more > > > *********************************************************** > # This needs two +1s in order to be committed to branch-1.7.0 > > > Diffs > ----- > > > ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/package/scripts/tez.py > 62e3cd6 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml > 93ad95d > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-env.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml > c1e8fa6 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/configuration/tez-site.xml > dfedb6b > > Diff: https://reviews.apache.org/r/26591/diff/ > > > Testing > ------- > > Deployed an HDP 2.2.1 cluster and switched the execution engine to tez. > Then changed the following properties to coincide with the changed files. > > HDFS => hadoop-env = > { > ... > > if [[ -d "/usr/hdp/current/tez-client" && -d "/etc/tez/conf/" ]]; then > # When using versioned RPMs, the tez-client will be a symlink to the > current folder of tez in HDP. > export > HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:/usr/hdp/current/tez-client/:/usr/hdp/current/tez-client/lib/:/etc/tez/conf/ > fi > > # Setting path to hdfs command line > export HADOOP_LIBEXEC_DIR={{hadoop_libexec_dir}} > > # Mostly required for hadoop 2.0 > export > JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64 > > ... > } > > > Hive => hive-env = > { > ... > > if [ "${HIVE_AUX_JARS_PATH}" != "" ]; then > export HIVE_AUX_JARS_PATH=${HIVE_AUX_JARS_PATH} > elif [ -d "/usr/hdp/current/hive-webhcat/share/hcatalog" ]; then > export HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog > fi > > ... > } > > Tez => tez.lib.uris = hdfs:///apps/tez/tez.tar.gz > > > After restarting the services, ran the following commands to simulate the > behavior of the scripts during the installation of Hadoop. > > export TEZ_CONF_DIR=/etc/tez/conf > export > TEZ_JARS=/usr/hdp/current/tez-client/*:/usr/hdp/current/tez-client/lib/* > export HADOOP_CLASSPATH=$TEZ_CONF_DIR:$TEZ_JARS:$HADOOP_CLASSPATH > > Then ran > su - hive > hive > > And was able to get the hive shell to start. > To test this on a brand new cluster install, would like for this to be > committed first. > > Also ran unit tests, > ---------------------------------------------------------------------- > Total run:641 > Total errors:0 > Total failures:0 > OK > > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 23:21.678s > [INFO] Finished at: Fri Oct 10 19:39:55 PDT 2014 > [INFO] Final Memory: 63M/1540M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Alejandro Fernandez > >
