-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47328/#review133146
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 166)
<https://reviews.apache.org/r/47328/#comment197443>

    fancy, but this is a constant that we don;t have to calculate every time, 
why not just say
    
    MEM_THRESHOLD = 33554432 # 32GB in KB



ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py (line 
2041)
<https://reviews.apache.org/r/47328/#comment197444>

    should HAWQ/PXF tests be now moved to separate service advisor tests files, 
perhaps as a separate effort ?


- Alexander Denissov


On May 13, 2016, 12:58 a.m., Matt wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47328/
> -----------------------------------------------------------
> 
> (Updated May 13, 2016, 12:58 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, and Lav Jain.
> 
> 
> Bugs: AMBARI-16646
>     https://issues.apache.org/jira/browse/AMBARI-16646
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Consider all HAWQ host machines. 
> If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory 
> as 2, else use 1
> 
> 
> Diffs
> -----
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  a26a398 
>   ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> be0f3e4 
> 
> Diff: https://reviews.apache.org/r/47328/diff/
> 
> 
> Testing
> -------
> 
> Added unit tests.
> 
> ```
> bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/
> bash-3.2$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ServiceAdvisor implementation for service 
> PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ServiceAdvisor implementation for 
> service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... 
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE ... 
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_getComponentLayoutValidations_nohawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test NO warning is generated when PXF is co-located with NAMENODE and 
> DATANODE ... ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE ... 
> ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ServiceAdvisor implementation for service PXF was loaded
> ServiceAdvisor implementation for service PXF was loaded
> ok
> test_getComponentLayoutValidations_sparkts_no_hive 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is not installed ... ok
> test_getComponentLayoutValidations_sparkts_with_hive 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendFalconConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendHAWQConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_recommendHBASEConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendHiveConfigurations_with_atlas 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendKAFKAConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendLogsearchConfiguration 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendRangerConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendRangerKMSConfigurations 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendSqoopConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendStormConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendTezConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendYARNConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_validateHAWQHdfsClientConfigurations 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ok
> test_validateHAWQSiteConfigurations 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ok
> test_validateHDFSConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_validateHiveConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> 
> ----------------------------------------------------------------------
> Ran 37 tests in 0.298s
> 
> OK
> bash-3.2$ cd ../HAWQ
> bash-3.2$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-05-12 17:56:31,989 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-05-12 17:56:31,994 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-05-12 17:56:31,998 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
> Kerberized ... 2016-05-12 17:56:32,006 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
> Kerberized ... 2016-05-12 17:56:32,012 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
> Kerberized ... 2016-05-12 17:56:32,019 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 2: Non HDFS-HA, Standalone Resource Management, 
> Kerberized ... 2016-05-12 17:56:32,028 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case3 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 3: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
> Not Kerberized ... 2016-05-12 17:56:32,035 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case4 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 4: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
> Kerberized ... 2016-05-12 17:56:32,043 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case5 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 5: Non HDFS-HA, YARN Resource Management YARN_HA, Not 
> Kerberized ... 2016-05-12 17:56:32,049 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case6 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, 
> Kerberized ... 2016-05-12 17:56:32,055 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-05-12 17:56:32,061 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, 
> Kerberized ... 2016-05-12 17:56:32,070 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not 
> Kerberized ... 2016-05-12 17:56:32,078 - Executing HAWQ Check ...
> ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-05-12 
> 17:56:32,086 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys 
> is either set to false or is not available in hawq-env.xml
> ok
> test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster)
> Run Stop HAWQ Cluster Immediate Mode ... ok
> test_stop_default (test_hawqmaster.TestHawqMaster)
> Run Stop HAWQMASTER ... ok
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_getServiceConfigurationRecommendations 
> (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok
> test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok
> 
> ----------------------------------------------------------------------
> Ran 51 tests in 0.226s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Matt
> 
>

Reply via email to