----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42610/#review118046 -----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/checks/HardcodedStackVersionPropertiesCheck.java (line 118) <https://reviews.apache.org/r/42610/#comment179317> Great check! - Alejandro Fernandez On Feb. 5, 2016, 5:41 p.m., Dmitro Lisnichenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42610/ > ----------------------------------------------------------- > > (Updated Feb. 5, 2016, 5:41 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan > Hurley, Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-14764 > https://issues.apache.org/jira/browse/AMBARI-14764 > > > Repository: ambari > > > Description > ------- > > # MapReduce jobs were failing after upgrading and restarting MapReduce2 > service (ClassNotFound exception). Also Oozie job were failing because of the > same reason. > # This was caused because mapreduce.application.classpath property had the > older version of the LZO codec > (/usr/hdp/current/hadoop/lib/hadoop-lzo-0.6.0.<OLD-STACK-VERSION>.jar). > # During upgrade the path was valid because we had not switched > /usr/hdp/current/hadoop symlink to new version. After upgrade, once the > symlink is updated to new stack version and after restarting MapReduce2 this > path becomes invalid. > # To fix this updated the LZO codec path to > /usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar in > mapreduce.application.classpath config property. Also in oozie-env property > added export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS” > > Fix: > 1. During stack upgrade, update lzo codec path in > mapreduce.application.classpath to > {code} > /usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar > {code} > > Also, looks like we also need to update following property for TEZ. > > {code} > <property> > <name>tez.cluster.additional.classpath.prefix</name> > <value>/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf/secure</value> > <description></description> > </property> > {code} > > 2. Update oozie-env and add > {code} > export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS" > {code} > > 3. Add upgrade pre-checks to validate if any config properties have > old-stack-version hardcodings and report these config properties to be > updated. > (attached screenshot) > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java > 14f819e > > ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java > a8c0525 > > ambari-server/src/main/java/org/apache/ambari/server/checks/HardcodedStackVersionPropertiesCheck.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixLzoCodecPath.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/OozieConfigCalculation.java > PRE-CREATION > > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml > 3a3395a > > ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-env.xml > 93742bb > > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml > e581506 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/OOZIE/configuration/oozie-env.xml > f0621cd > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml > 098682b > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml > fe9d97c > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml > b756deb > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml > 282d143 > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml > 133efae > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml > 39756d9 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml > e3b8546 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml > 9b36ec9 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml > a0c16a4 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml > 660dbe2 > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml > 2600eae > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml > 02783e9 > > ambari-server/src/test/java/org/apache/ambari/server/checks/HardcodedStackVersionPropertiesCheckTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixLzoCodecPathTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/OozieConfigCalculationTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/42610/diff/ > > > Testing > ------- > > - Manual check of 2.3->2.4 EU upgrade on live cluster > - Manual check of 2.3->2.4 EU upgrade on live cluster > - Manual verification of a pre-upgrade check > > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Views ...................................... SUCCESS [2.012s] > [INFO] Ambari Metrics Common ............................. SUCCESS [0.886s] > [INFO] Ambari Server ..................................... SUCCESS [1:01.074s] > [INFO] Ambari Agent ...................................... SUCCESS [12.373s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 1:19.469s > [INFO] Finished at: Thu Feb 04 20:07:24 EET 2016 > [INFO] Final Memory: 72M/1397M > > > File Attachments > ---------------- > > screenshot of pre-upgrade check dialog > > https://reviews.apache.org/media/uploaded/files/2016/01/21/05890946-df65-4724-bcdd-3450b159b102__scr1.png > > > Thanks, > > Dmitro Lisnichenko > >
