> On June 17, 2016, 8:31 a.m., Jonathan Hurley wrote: > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml, > > lines 159-164 > > <https://reviews.apache.org/r/48805/diff/1/?file=1421659#file1421659line159> > > > > This isn't really future-proof, is it? If I add something to the SPARK > > section, it won't be reflected here. I think that's a major problem with > > this workflow. > > Alejandro Fernandez wrote: > It's an artifact of the fact that RU today makes additional config > changes immediately before restarting the current service. > In practice, we only make these sort of config changes in the pre-upgrade > section, so perhaps we should refactor all RUs to save final configs all the > way up front, just like EU. > > I don't want to destabilize RU with such a big change right now. Will > drop.
I think that's my point - we've gotten this far with RU the way it is and are just seeing this issue surface now? Maybe we need to not fix it for this release and instead focus our efforts on modifying the configs differently. - Jonathan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48805/#review138206 ----------------------------------------------------------- On June 16, 2016, 1:47 p.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48805/ > ----------------------------------------------------------- > > (Updated June 16, 2016, 1:47 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Nate Cole, and Tim Thorpe. > > > Bugs: AMBARI-17280 > https://issues.apache.org/jira/browse/AMBARI-17280 > > > Repository: ambari > > > Description > ------- > > During RU, HiveServer2 is restarted but the newer tez configs have not yet > been saved, which is incorrect because Hive has a dependency on Tez. > This is important when configs change during a major stack upgrade, e.g., HDP > 2.4 -> 2.5. What happens today is, > > * Install packages generates /etc/tez/2.5.0.0-1/0 and copies the configs from > /etc/tez/2.4.0.0-1/0/ to the new folder > * If configs change during RU, then Hive is restarted and the classpath means > that it will pick up the older tez configs from the new /etc/tez/2.5.0.0-1/0 > folder > > > This problem exists for all of these components: > > * HiveServer: depends on Tez and MapReduce clients > * ATS: depends on Tez and Spark clients > * Oozie: depends on Tez, Spark, and MapReduce clients > > This problem only exists when configs change (so crossing major stack > version) and during RU (because it is allowed to change configs during the > middle of restarting services). > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > fb3ae69 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java > 80bb26c > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py > 63c72f7 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py > ef41453 > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py > 44239c7 > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez.py > 67466e3 > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py > c79d63b > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py > db22004 > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py > 90f885a > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py > d1ec15b > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml > 4187d64 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml > 3461ad4 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml > 426b452 > ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py e53eb4b > > Diff: https://reviews.apache.org/r/48805/diff/ > > > Testing > ------- > > Verified during RU from HDP 2.4 to 2.5 with ATS, Hive, Tez, Oozie, and Spark > > Python unit tests passed, > ---------------------------------------------------------------------- > Total run:1062 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Alejandro Fernandez > >
