-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47328/
-----------------------------------------------------------

Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam Tadi, 
jun aoki, and Lav Jain.


Bugs: AMBARI-16646
    https://issues.apache.org/jira/browse/AMBARI-16646


Repository: ambari


Description
-------

Consider all HAWQ host machines. 
If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory as 
2, else use 1


Diffs
-----

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
a26a398 
  ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py be0f3e4 

Diff: https://reviews.apache.org/r/47328/diff/


Testing
-------

Added unit tests.

```
bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/
bash-3.2$ python -m discover -v
test_createComponentLayoutRecommendations_hawq_1_Host 
(test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT does not get recommended during Add Service Wizard, when 
HAWQ has already been installed ... ServiceAdvisor implementation for service 
HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Cluster Install Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components ... ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF does not get recommended during Add Service Wizard, when PXF has 
already been installed ... ServiceAdvisor implementation for service PXF was 
loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Add Service Wizard, when PXF is 
selected for installation ... ServiceAdvisor implementation for service PXF was 
loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_createComponentLayoutRecommendations_pxf_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Cluster Install Wizard, when 
PXF is selected for installation ... ServiceAdvisor implementation for service 
PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test layout validations for HAWQ components on a 3-node cluster ... 
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
(test_stack_advisor.TestHDP23StackAdvisor)
Test validation warning for HAWQ segment not colocated with DATANODE ... 
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_nohawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components on a 3-node cluster ... ok
test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test NO warning is generated when PXF is co-located with NAMENODE and DATANODE 
... ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE or DATANODE 
... ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_pxf_not_co_located_with_nn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE ... 
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE or DATANODE 
... ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_sparkts_no_hive 
(test_stack_advisor.TestHDP23StackAdvisor)
Test SparkTS is picked when Hive is not installed ... ok
test_getComponentLayoutValidations_sparkts_with_hive 
(test_stack_advisor.TestHDP23StackAdvisor)
Test SparkTS is picked when Hive is installed ... ok
test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendFalconConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendHAWQConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_recommendHBASEConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ok
test_recommendHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ok
test_recommendHiveConfigurations_with_atlas 
(test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendKAFKAConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendLogsearchConfiguration (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendRangerConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendRangerKMSConfigurations 
(test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendSqoopConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendStormConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendTezConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ok
test_recommendYARNConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ok
test_validateHAWQHdfsClientConfigurations 
(test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
for service HAWQ was loaded
ok
test_validateHAWQSiteConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ServiceAdvisor implementation for service HAWQ was loaded
ok
test_validateHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ok
test_validateHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) ... 
ok

----------------------------------------------------------------------
Ran 37 tests in 0.298s

OK
bash-3.2$ cd ../HAWQ
bash-3.2$ python -m discover -v
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Remove HAWQ Standby ... 2016-05-12 17:56:31,989 - Removing 
HAWQ Standby Master ...
ok
test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Resync HAWQ Standby ... 2016-05-12 17:56:31,994 - HAWQ 
Standby Master Re-Sync started in fast mode...
ok
test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-05-12 17:56:31,998 - Executing HAWQ Check ...
ok
test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-05-12 17:56:32,006 - Executing HAWQ Check ...
ok
test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
Kerberized ... 2016-05-12 17:56:32,012 - Executing HAWQ Check ...
ok
test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
Kerberized ... 2016-05-12 17:56:32,019 - Executing HAWQ Check ...
ok
test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 2: Non HDFS-HA, Standalone Resource Management, 
Kerberized ... 2016-05-12 17:56:32,028 - Executing HAWQ Check ...
ok
test_run_hawq_check_case3 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 3: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
Not Kerberized ... 2016-05-12 17:56:32,035 - Executing HAWQ Check ...
ok
test_run_hawq_check_case4 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 4: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-05-12 17:56:32,043 - Executing HAWQ Check ...
ok
test_run_hawq_check_case5 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 5: Non HDFS-HA, YARN Resource Management YARN_HA, Not 
Kerberized ... 2016-05-12 17:56:32,049 - Executing HAWQ Check ...
ok
test_run_hawq_check_case6 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, 
Kerberized ... 2016-05-12 17:56:32,055 - Executing HAWQ Check ...
ok
test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-05-12 17:56:32,061 - Executing HAWQ Check ...
ok
test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized 
... 2016-05-12 17:56:32,070 - Executing HAWQ Check ...
ok
test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not 
Kerberized ... 2016-05-12 17:56:32,078 - Executing HAWQ Check ...
ok
test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-05-12 17:56:32,086 
- Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to 
false or is not available in hawq-env.xml
ok
test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster)
Run Stop HAWQ Cluster Immediate Mode ... ok
test_stop_default (test_hawqmaster.TestHawqMaster)
Run Stop HAWQMASTER ... ok
test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
test_install_default (test_hawqsegment.TestHawqSegment) ... ok
test_start_default (test_hawqsegment.TestHawqSegment) ... ok
test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
test_install_default (test_hawqstandby.TestHawqStandby) ... ok
test_start_default (test_hawqstandby.TestHawqStandby) ... ok
test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
test_getServiceConfigurationRecommendations 
(test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok
test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok

----------------------------------------------------------------------
Ran 51 tests in 0.226s

OK
```


Thanks,

Matt

Reply via email to