Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ

2016-05-13 Thread bhuvnesh chaudhary

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47328/#review133178
---


Ship it!




Ship It!

- bhuvnesh chaudhary


On May 13, 2016, 6:59 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47328/
> ---
> 
> (Updated May 13, 2016, 6:59 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, and Lav Jain.
> 
> 
> Bugs: AMBARI-16646
> https://issues.apache.org/jira/browse/AMBARI-16646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Consider all HAWQ host machines. 
> If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory 
> as 2, else use 1
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  a26a398 
>   ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> be0f3e4 
> 
> Diff: https://reviews.apache.org/r/47328/diff/
> 
> 
> Testing
> ---
> 
> Added unit tests.
> 
> ```
> bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/
> bash-3.2$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service 

Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ

2016-05-13 Thread Matt


> On May 13, 2016, 10:08 a.m., Alexander Denissov wrote:
> > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py, line 
> > 2041
> > 
> >
> > should HAWQ/PXF tests be now moved to separate service advisor tests 
> > files, perhaps as a separate effort ?

Refer https://issues.apache.org/jira/browse/AMBARI-16647


> On May 13, 2016, 10:08 a.m., Alexander Denissov wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py,
> >  line 166
> > 
> >
> > fancy, but this is a constant that we don;t have to calculate every 
> > time, why not just say
> > 
> > MEM_THRESHOLD = 33554432 # 32GB in KB

Will update


- Matt


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47328/#review133146
---


On May 12, 2016, 5:58 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47328/
> ---
> 
> (Updated May 12, 2016, 5:58 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, and Lav Jain.
> 
> 
> Bugs: AMBARI-16646
> https://issues.apache.org/jira/browse/AMBARI-16646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Consider all HAWQ host machines. 
> If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory 
> as 2, else use 1
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  a26a398 
>   ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> be0f3e4 
> 
> Diff: https://reviews.apache.org/r/47328/diff/
> 
> 
> Testing
> ---
> 
> Added unit tests.
> 
> ```
> bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/
> bash-3.2$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> 

Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ

2016-05-13 Thread Alexander Denissov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47328/#review133154
---


Ship it!




Ship It!

- Alexander Denissov


On May 13, 2016, 12:58 a.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47328/
> ---
> 
> (Updated May 13, 2016, 12:58 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, and Lav Jain.
> 
> 
> Bugs: AMBARI-16646
> https://issues.apache.org/jira/browse/AMBARI-16646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Consider all HAWQ host machines. 
> If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory 
> as 2, else use 1
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  a26a398 
>   ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> be0f3e4 
> 
> Diff: https://reviews.apache.org/r/47328/diff/
> 
> 
> Testing
> ---
> 
> Added unit tests.
> 
> ```
> bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/
> bash-3.2$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service 

Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ

2016-05-13 Thread Alexander Denissov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47328/#review133146
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 166)


fancy, but this is a constant that we don;t have to calculate every time, 
why not just say

MEM_THRESHOLD = 33554432 # 32GB in KB



ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py (line 
2041)


should HAWQ/PXF tests be now moved to separate service advisor tests files, 
perhaps as a separate effort ?


- Alexander Denissov


On May 13, 2016, 12:58 a.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47328/
> ---
> 
> (Updated May 13, 2016, 12:58 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, and Lav Jain.
> 
> 
> Bugs: AMBARI-16646
> https://issues.apache.org/jira/browse/AMBARI-16646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Consider all HAWQ host machines. 
> If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory 
> as 2, else use 1
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  a26a398 
>   ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> be0f3e4 
> 
> Diff: https://reviews.apache.org/r/47328/diff/
> 
> 
> Testing
> ---
> 
> Added unit tests.
> 
> ```
> bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/
> bash-3.2$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
>