----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25265/#review52058 -----------------------------------------------------------
Ship it! Ship It! - Dmitro Lisnichenko On Sept. 2, 2014, 6:42 p.m., Andrew Onischuk wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/25265/ > ----------------------------------------------------------- > > (Updated Sept. 2, 2014, 6:42 p.m.) > > > Review request for Ambari and Dmitro Lisnichenko. > > > Bugs: AMBARI-7119 > https://issues.apache.org/jira/browse/AMBARI-7119 > > > Repository: ambari > > > Description > ------- > > PROBLEM: log4j settings made via Ambari update the log4j file but do not take > any affect when restarting HDFS. It seems there are hardcoded settings in > /usr/lib/hadoop/libexec/hadoop-config.sh such as this at line 221: > > HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.root.logger=$ > > {HADOOP_ROOT_LOGGER:-INFO,console} > > " > > BUSINESS IMPACT: Customers have to change core files or set environment > variables explicitly by setting up a profile script > > STEPS TO REPRODUCE: Log in to Ambari and change log4j properties such that > hadoop.root.logger=INFO,DRFA. The log4j file is updated in /etc/hadoop/conf. > > Restart the HDFS service. Do a ps -ef | grep <PID> for the namenode. The > process shows duplicate entries for several properties and does not show the > logging change. Here is the duplication and incorrect root logger setting seen > locally in testing: > > hdfs 4304 1 14 07:26 ? 00:00:10 /usr/jdk64/jdk1.7.0_45/bin/java > -Dproc_namenode -Xmx1024m -Djava.net.preferIPv4Stack=true > -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true > -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop.log > -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs > -Dhadoop.root.logger=INFO,console > -Djava.library.path=:/usr/lib/hadoop/lib/native/Linux- > amd64-64:/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml > -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/hdfs > -Dhadoop.log.file=hadoop-hdfs-namenode-sandbox.hortonworks.com.log > -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs > -Dhadoop.root.logger=INFO,RFA -Djava.library.path=:/usr/lib/hadoop/lib/native > /Linux-amd64-64:/usr/lib/hadoop/lib/native:/usr/lib/hadoop/lib/native/Linux- > amd64-64:/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native > -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server > -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC > -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=100m > -XX:MaxNewSize=50m -Xloggc:/var/log/hadoop/hdfs/gc.log-201407140726 > -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps > -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS > -Dhdfs.audit.logger=INFO,DRFAAUDIT -server -XX:ParallelGCThreads=8 > -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log > -XX:NewSize=100m -XX:MaxNewSize=50m > -Xloggc:/var/log/hadoop/hdfs/gc.log-201407140726 -verbose:gc > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xms1024m > -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS > -Dhdfs.audit.logger=INFO,DRFAAUDIT -server -XX:ParallelGCThreads=8 > -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hadoop/hdfs/hs_err_pid%p.log > -XX:NewSize=100m -XX:MaxNewSize=50m > -Xloggc:/var/log/hadoop/hdfs/gc.log-201407140726 -verbose:gc > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xms1024m > -Xmx1024m -Dhadoop.security.logger=INFO,DRFA > > ACTUAL BEHAVIOR: log4j changes made in Ambari do not persist in process. It > seems there are values set in /usr/lib/hadoop/libexec/hadoop-config.sh that > override no matter what. There is also duplication of settings, assuming this > is from hadoop-config.sh as well. > > EXPECTED BEHAVIOR: Settings made in Ambari should be persisted and used by the > process. > > SUPPORT ANALYSIS: Support made changes to log4j in Ambari on a test cluster > and they were not used > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java > 0ac9da7 > > ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/params.py > a4fb172 > > ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/hadoop-env.xml > ba704ba > > ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/configuration/hdfs-log4j.xml > 1366c5d > > ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/params.py > 58f01f8 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py > 3fe5905 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/configuration/hadoop-env.xml > 573a67c > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/configuration/hdfs-log4j.xml > 976a3c2 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/params.py > 5c2f792 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java > e2ce75e > ambari-web/app/data/HDP2/site_properties.js 3ba0246 > ambari-web/app/data/site_properties.js 1f6bb63 > > Diff: https://reviews.apache.org/r/25265/diff/ > > > Testing > ------- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
