> On June 17, 2016, 12:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml, 
> > lines 159-164
> > <https://reviews.apache.org/r/48805/diff/1/?file=1421659#file1421659line159>
> >
> >     This isn't really future-proof, is it? If I add something to the SPARK 
> > section, it won't be reflected here. I think that's a major problem with 
> > this workflow.
> 
> Alejandro Fernandez wrote:
>     It's an artifact of the fact that RU today makes additional config 
> changes immediately before restarting the current service.
>     In practice, we only make these sort of config changes in the pre-upgrade 
> section, so perhaps we should refactor all RUs to save final configs all the 
> way up front, just like EU.
>     
>     I don't want to destabilize RU with such a big change right now. Will 
> drop.
> 
> Jonathan Hurley wrote:
>     I think that's my point - we've gotten this far with RU the way it is and 
> are just seeing this issue surface now? Maybe we need to not fix it for this 
> release and instead focus our efforts on modifying the configs differently.

It's actually a bug when HS2, Oozie Server, and ATS start up using wrong 
configs. I prefer this fix in Ambari 2.4.0 and revisit in 2.5.0 for a more 
generic approach.


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48805/#review138206
-----------------------------------------------------------


On June 21, 2016, 1:12 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48805/
> -----------------------------------------------------------
> 
> (Updated June 21, 2016, 1:12 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17280
>     https://issues.apache.org/jira/browse/AMBARI-17280
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> During RU, HiveServer2 is restarted but the newer tez configs have not yet 
> been saved, which is incorrect because Hive has a dependency on Tez.
> This is important when configs change during a major stack upgrade, e.g., HDP 
> 2.4 -> 2.5. What happens today is,
> 
> * Install packages generates /etc/tez/2.5.0.0-1/0 and copies the configs from 
> /etc/tez/2.4.0.0-1/0/ to the new folder
> * If configs change during RU, then Hive is restarted and the classpath means 
> that it will pick up the older tez configs from the new /etc/tez/2.5.0.0-1/0 
> folder
> 
> 
> This problem exists for all of these components:
> 
> * HiveServer: depends on Tez and MapReduce clients
> * ATS: depends on Tez and Spark clients
> * Oozie: depends on Tez, Spark, and MapReduce clients
> 
> This problem only exists when configs change (so crossing major stack 
> version) and during RU (because it is allowed to change configs during the 
> middle of restarting services).
> 
> 
> Diffs
> -----
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  b994fce 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 49dcb4e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
>  80bb26c 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py
>  63c72f7 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py
>  ef41453 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py
>  44239c7 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez.py
>  67466e3 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py
>  c79d63b 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
>  db22004 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  90f885a 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  d1ec15b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 4187d64 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 3461ad4 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 426b452 
>   ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py e53eb4b 
> 
> Diff: https://reviews.apache.org/r/48805/diff/
> 
> 
> Testing
> -------
> 
> Verified during RU from HDP 2.4 to 2.5 with ATS, Hive, Tez, Oozie, and Spark
> 
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:1062
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Reply via email to