Re: Review Request 63535: Send ClusterSummary to install_packages and conf-select exclusion

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/#review190052
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Nov. 3, 2017, 4:59 a.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63535/
> ---
> 
> (Updated Nov. 3, 2017, 4:59 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-22360
> https://issues.apache.org/jira/browse/AMBARI-22360
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pass the ClusterVersionSummary object with install_packages.
> 
> - {{conf-select}} should be called as normal for {{STANDARD}} upgrades and 
> component installations / host installs
> - {{conf-select}} should only be invoked for the services involved in a 
> {{PATCH}}/{{MAINT}} upgrade
> -- The exception here is that hadoop shared components (HDFS, YARN, MapR) can 
> never be {{conf-select}}'d in this case.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c89e7675a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  b3f91304c8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  a20658304f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
>  b68dc22093 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java
>  e9d99208e5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java
>  29505c832e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
>  7944de8e8c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  19019502dd 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 
> 8ba22d263b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
>  20b12a97e5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
>  51979f6dee 
>   ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 
> 92dd634cca 
> 
> 
> Diff: https://reviews.apache.org/r/63535/diff/1/
> 
> 
> Testing
> ---
> 
> Manual.  Unit tests:
> 
> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 4922, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 24:46.046s
> [INFO] Finished at: Thu Nov 02 23:22:42 EDT 2017
> [INFO] Final Memory: 85M/1940M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 63535: Send ClusterSummary to install_packages and conf-select exclusion

[Nate Cole]

> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
> > Lines 310-312 (patched)
> > 
> >
> > Should we allow this method to just be "the method" to use to get 
> > packages whether or not you're in an upgrade? That way, the dependendant 
> > code can just ask it - it would return all packages in the event that there 
> > is no PATCH/MAINT ...

Wasn't really interested in changing code that is working already when this is 
only for the install_packages flow.


> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
> > Lines 321-323 (patched)
> > 
> >
> > Just trying to think if this is truly a warning. If iterating over a 
> > cluster where you have non-upgradable services, would this warning get 
> > printed out? It's fine to say that for Service X, there are no packages ... 
> > but should it be a warning?

ok


> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
> > Lines 416 (patched)
> > 
> >
> > Should this string be a part of KeyNames, so that its centralized?

ok


> On Nov. 3, 2017, 10:19 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
> > Lines 1190 (patched)
> > 
> >
> > Do you need to adjust the stack_packages for BigInsights as well?
> > 
> > Also - when you cherry pick, don't forget that there is a stack 
> > packages under HDP 3.0 in trunk...

To my knowledge, IOP clusters aren't candidates for patching and would never 
get this structure.  So no.

Ok on 3.0


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/#review190028
---


On Nov. 2, 2017, 10:59 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63535/
> ---
> 
> (Updated Nov. 2, 2017, 10:59 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-22360
> https://issues.apache.org/jira/browse/AMBARI-22360
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pass the ClusterVersionSummary object with install_packages.
> 
> - {{conf-select}} should be called as normal for {{STANDARD}} upgrades and 
> component installations / host installs
> - {{conf-select}} should only be invoked for the services involved in a 
> {{PATCH}}/{{MAINT}} upgrade
> -- The exception here is that hadoop shared components (HDFS, YARN, MapR) can 
> never be {{conf-select}}'d in this case.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c89e7675a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  b3f91304c8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  a20658304f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
>  b68dc22093 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java
>  e9d99208e5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java
>  29505c832e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
>  7944de8e8c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  19019502dd 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 
> 8ba22d263b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
>  20b12a97e5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
>  51979f6dee 
>   ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 
> 92dd634cca 
> 
> 
> Diff: https://reviews.apache.org/r/63535/diff/1/
> 
> 
> Testing
> ---
> 
> Manual.  Unit tests:
> 
> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 4922, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> 

Re: Review Request 63535: Send ClusterSummary to install_packages and conf-select exclusion

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/#review190028
---


Fix it, then Ship it!





ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Lines 310-312 (patched)


Should we allow this method to just be "the method" to use to get packages 
whether or not you're in an upgrade? That way, the dependendant code can just 
ask it - it would return all packages in the event that there is no PATCH/MAINT 
...



ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
Lines 321-323 (patched)


Just trying to think if this is truly a warning. If iterating over a 
cluster where you have non-upgradable services, would this warning get printed 
out? It's fine to say that for Service X, there are no packages ... but should 
it be a warning?



ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
Lines 416 (patched)


Should this string be a part of KeyNames, so that its centralized?



ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
Lines 1190 (patched)


Do you need to adjust the stack_packages for BigInsights as well?

Also - when you cherry pick, don't forget that there is a stack packages 
under HDP 3.0 in trunk...


- Jonathan Hurley


On Nov. 2, 2017, 10:59 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63535/
> ---
> 
> (Updated Nov. 2, 2017, 10:59 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-22360
> https://issues.apache.org/jira/browse/AMBARI-22360
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pass the ClusterVersionSummary object with install_packages.
> 
> - {{conf-select}} should be called as normal for {{STANDARD}} upgrades and 
> component installations / host installs
> - {{conf-select}} should only be invoked for the services involved in a 
> {{PATCH}}/{{MAINT}} upgrade
> -- The exception here is that hadoop shared components (HDFS, YARN, MapR) can 
> never be {{conf-select}}'d in this case.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c89e7675a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  b3f91304c8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  a20658304f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
>  b68dc22093 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java
>  e9d99208e5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java
>  29505c832e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
>  7944de8e8c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  19019502dd 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 
> 8ba22d263b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
>  20b12a97e5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
>  51979f6dee 
>   ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 
> 92dd634cca 
> 
> 
> Diff: https://reviews.apache.org/r/63535/diff/1/
> 
> 
> Testing
> ---
> 
> Manual.  Unit tests:
> 
> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 4922, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 24:46.046s
> [INFO] Finished at: Thu Nov 02 23:22:42 EDT 2017
> [INFO] Final Memory: 85M/1940M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Review Request 63535: Send ClusterSummary to install_packages and conf-select exclusion

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63535/
---

Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley.


Bugs: AMBARI-22360
https://issues.apache.org/jira/browse/AMBARI-22360


Repository: ambari


Description
---

Pass the ClusterVersionSummary object with install_packages.

- {{conf-select}} should be called as normal for {{STANDARD}} upgrades and 
component installations / host installs
- {{conf-select}} should only be invoked for the services involved in a 
{{PATCH}}/{{MAINT}} upgrade
-- The exception here is that hadoop shared components (HDFS, YARN, MapR) can 
never be {{conf-select}}'d in this case.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 c89e7675a4 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 b3f91304c8 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 a20658304f 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
 b68dc22093 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ClusterVersionSummary.java
 e9d99208e5 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ServiceVersionSummary.java
 29505c832e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
 7944de8e8c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 19019502dd 
  ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 
8ba22d263b 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
 20b12a97e5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 51979f6dee 
  ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py 
92dd634cca 


Diff: https://reviews.apache.org/r/63535/diff/1/


Testing
---

Manual.  Unit tests pending.


Thanks,

Nate Cole