> On June 17, 2016, 12:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py,
> >  line 55
> > <https://reviews.apache.org/r/48805/diff/1/?file=1421656#file1421656line55>
> >
> >     Is there any way to combine this logic for the clients? Seems like a 
> > lot of copy/paste

Only some clients have dependencies, I can come up with a function in 
common-services.


> On June 17, 2016, 12:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml, 
> > lines 159-164
> > <https://reviews.apache.org/r/48805/diff/1/?file=1421659#file1421659line159>
> >
> >     This isn't really future-proof, is it? If I add something to the SPARK 
> > section, it won't be reflected here. I think that's a major problem with 
> > this workflow.

It's an artifact of the fact that RU today makes additional config changes 
immediately before restarting the current service.
In practice, we only make these sort of config changes in the pre-upgrade 
section, so perhaps we should refactor all RUs to save final configs all the 
way up front, just like EU.

I don't want to destabilize RU with such a big change right now. Will drop.


> On June 17, 2016, 12:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml, 
> > lines 188-193
> > <https://reviews.apache.org/r/48805/diff/1/?file=1421661#file1421661line188>
> >
> >     I'm a little worried about this kind of logic. You're saying that Oozie 
> > requires tez-site configs to be updated. Yet, the execute-stage is scoped 
> > for a TEZ/TEZ_CLIENT. This actually needs to run on Oozie, no?

This needs to run on all Tez Clients. If an Oozie Server also has Tez Client, 
then today it will have the wrong configs.
If the customer has Tez but no Oozie, then there's no need to run this step.

Will drop.


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48805/#review138206
-----------------------------------------------------------


On June 16, 2016, 5:47 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48805/
> -----------------------------------------------------------
> 
> (Updated June 16, 2016, 5:47 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17280
>     https://issues.apache.org/jira/browse/AMBARI-17280
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> During RU, HiveServer2 is restarted but the newer tez configs have not yet 
> been saved, which is incorrect because Hive has a dependency on Tez.
> This is important when configs change during a major stack upgrade, e.g., HDP 
> 2.4 -> 2.5. What happens today is,
> 
> * Install packages generates /etc/tez/2.5.0.0-1/0 and copies the configs from 
> /etc/tez/2.4.0.0-1/0/ to the new folder
> * If configs change during RU, then Hive is restarted and the classpath means 
> that it will pick up the older tez configs from the new /etc/tez/2.5.0.0-1/0 
> folder
> 
> 
> This problem exists for all of these components:
> 
> * HiveServer: depends on Tez and MapReduce clients
> * ATS: depends on Tez and Spark clients
> * Oozie: depends on Tez, Spark, and MapReduce clients
> 
> This problem only exists when configs change (so crossing major stack 
> version) and during RU (because it is allowed to change configs during the 
> middle of restarting services).
> 
> 
> Diffs
> -----
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
>  80bb26c 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py
>  63c72f7 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py
>  ef41453 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py
>  44239c7 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez.py
>  67466e3 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py
>  c79d63b 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
>  db22004 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  90f885a 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  d1ec15b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 4187d64 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 3461ad4 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 426b452 
>   ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py e53eb4b 
> 
> Diff: https://reviews.apache.org/r/48805/diff/
> 
> 
> Testing
> -------
> 
> Verified during RU from HDP 2.4 to 2.5 with ATS, Hive, Tez, Oozie, and Spark
> 
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:1062
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Reply via email to