----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48805/#review138847 -----------------------------------------------------------
I understand what you're doing here, but it's a lot of work for something that we are going to throw away anyway after 2.4 - is there no way to simply define multiple groupings so that we can still isolate the config changes by service? ambari-common/src/main/python/resource_management/libraries/script/script.py (line 158) <https://reviews.apache.org/r/48805/#comment204041> This function returns a value - it doesn't _do_ anything ... can we rename it to reflect that? ambari-common/src/main/python/resource_management/libraries/script/script.py (lines 168 - 171) <https://reviews.apache.org/r/48805/#comment204039> Way too easy to break this by simply renames in params.py ... can we extract these and make them constants somewhere? ambari-common/src/main/python/resource_management/libraries/script/script.py (line 175) <https://reviews.apache.org/r/48805/#comment204040> no need for `if params.version` ... the `check_stack_feature` already does that. ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java (lines 104 - 111) <https://reviews.apache.org/r/48805/#comment204042> What about the other types of groupings, like Colocated? ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml (line 153) <https://reviews.apache.org/r/48805/#comment204043> The SERVICE/COMPONENT string is too easy to mess up; we should have distinct service and component attributes - Jonathan Hurley On June 20, 2016, 9:12 p.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48805/ > ----------------------------------------------------------- > > (Updated June 20, 2016, 9:12 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Nate Cole, and Tim Thorpe. > > > Bugs: AMBARI-17280 > https://issues.apache.org/jira/browse/AMBARI-17280 > > > Repository: ambari > > > Description > ------- > > During RU, HiveServer2 is restarted but the newer tez configs have not yet > been saved, which is incorrect because Hive has a dependency on Tez. > This is important when configs change during a major stack upgrade, e.g., HDP > 2.4 -> 2.5. What happens today is, > > * Install packages generates /etc/tez/2.5.0.0-1/0 and copies the configs from > /etc/tez/2.4.0.0-1/0/ to the new folder > * If configs change during RU, then Hive is restarted and the classpath means > that it will pick up the older tez configs from the new /etc/tez/2.5.0.0-1/0 > folder > > > This problem exists for all of these components: > > * HiveServer: depends on Tez and MapReduce clients > * ATS: depends on Tez and Spark clients > * Oozie: depends on Tez, Spark, and MapReduce clients > > This problem only exists when configs change (so crossing major stack > version) and during RU (because it is allowed to change configs during the > middle of restarting services). > > > Diffs > ----- > > > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py > 4eb0015 > > ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py > b994fce > > ambari-common/src/main/python/resource_management/libraries/script/script.py > 49dcb4e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > fb3ae69 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java > 80bb26c > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py > 63c72f7 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py > ef41453 > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py > 44239c7 > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez.py > 67466e3 > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py > c79d63b > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py > db22004 > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py > 90f885a > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py > d1ec15b > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml > 4187d64 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml > 3461ad4 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml > 426b452 > ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py e53eb4b > > Diff: https://reviews.apache.org/r/48805/diff/ > > > Testing > ------- > > Verified during RU from HDP 2.4 to 2.5 with ATS, Hive, Tez, Oozie, and Spark > > Python unit tests passed, > ---------------------------------------------------------------------- > Total run:1062 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Alejandro Fernandez > >
