Re: Review Request 51754: AMBARI-18337 Syntax Error in Ambari HAWQ Unit test with Python 2.6
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51754/#review149208
---
Ship it!
Ship It!
- Alexander Denissov
On Sept. 9, 2016, 1:21 a.m., Masahiro Tanaka wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51754/
> ---
>
> (Updated Sept. 9, 2016, 1:21 a.m.)
>
>
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
>
>
> Bugs: AMBARI-18337
> https://issues.apache.org/jira/browse/AMBARI-18337
>
>
> Repository: ambari
>
>
> Description
> ---
>
> With Python 2.6, ` {"A", "B"} ` syntax isn't allowed.
>
> Ref. https://docs.python.org/2/library/stdtypes.html#set-types-set-frozenset
>
> ```
> As of Python 2.7, non-empty sets (not frozensets) can be created by placing a
> comma-separated list of elements within braces, for example: {'jack',
> 'sjoerd'}, in addition to the set constructor.
> ```
>
> Error
>
> ```
> Traceback (most recent call last):
> File "unitTests.py", line 129, in stack_test_executor
> modules]
> File "/usr/lib64/python2.6/unittest.py", line 575, in loadTestsFromName
> module = __import__('.'.join(parts_copy))
> File
> "/tmp/ambari/ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py",
> li
> ne 646
> self.assertFalse({'HAWQMASTER', 'HAWQSTANDBY'}.issubset(hostComponents))
> ^
> SyntaxError: invalid syntax
> ```
>
>
> Diffs
> -
>
> ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py
> 71c109f
>
> Diff: https://reviews.apache.org/r/51754/diff/
>
>
> Testing
> ---
>
> +1 overall. Here are the results of testing the latest attachment
> http://issues.apache.org/jira/secure/attachment/12827665/AMBARI-18337.patch
> against trunk revision .
>
> +1 @author. The patch does not contain any @author tags.
>
> +1 tests included. The patch appears to include 1 new or modified test files.
>
> +1 javac. The applied patch does not increase the total number of javac
> compiler warnings.
>
> +1 release audit. The applied patch does not increase the total number of
> release audit warnings.
>
> +1 core tests. The patch passed unit tests in ambari-server.
>
> Test results:
> https://builds.apache.org/job/Ambari-trunk-test-patch/8623//testReport/
> Console output:
> https://builds.apache.org/job/Ambari-trunk-test-patch/8623//console
>
> This message is automatically generated.
>
>
> Thanks,
>
> Masahiro Tanaka
>
>
Re: Review Request 50965: Ambari unit tests for HAWQ are not being called
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50965/#review145407 --- Ship it! Looks good, very useful refactoring. Please consider a few suggestions for naming. - Alexander Denissov On Aug. 10, 2016, 8:30 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50965/ > --- > > (Updated Aug. 10, 2016, 8:30 p.m.) > > > Review request for Ambari, Alexander Denissov and Matt. > > > Bugs: AMBARI-18110 > https://issues.apache.org/jira/browse/AMBARI-18110 > > > Repository: ambari > > > Description > --- > > Ambari build jobs are not running the unit tests for HAWQ. > HAWQ (and PXF) unit tests in the new common-services directory under test is > being ignored during mvn's test phase. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 529c492 > > ambari-server/src/test/python/common-services/HAWQ/test_alert_component_status.py > b2e1d4d > > ambari-server/src/test/python/common-services/HAWQ/test_alert_segment_registration_status.py > 4066e29 > > ambari-server/src/test/python/common-services/HAWQ/test_alert_sync_status.py > a44cad1 > ambari-server/src/test/python/common-services/HAWQ/test_hawqcommon.py > PRE-CREATION > ambari-server/src/test/python/common-services/HAWQ/test_hawqmaster.py > 62d54cf > ambari-server/src/test/python/common-services/HAWQ/test_hawqsegment.py > 48203ca > ambari-server/src/test/python/common-services/HAWQ/test_hawqstandby.py > 8674549 > ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py > e0f1c43 > ambari-server/src/test/python/common-services/PXF/test_alerts_api_status.py > ee187e2 > ambari-server/src/test/python/common-services/PXF/test_service_advisor.py > 8b2a4ba > ambari-server/src/test/python/unitTests.py b010804 > > Diff: https://reviews.apache.org/r/50965/diff/ > > > Testing > --- > > ``` > Running tests for stack:None service:HAWQ > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Remove HAWQ Standby ...
Re: Review Request 50965: Ambari unit tests for HAWQ are not being called
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50965/#review145405 --- ambari-server/src/test/python/common-services/HAWQ/test_alert_component_status.py (line 35) <https://reviews.apache.org/r/50965/#comment211628> probably do not need this here ambari-server/src/test/python/common-services/HAWQ/test_hawqcommon.py (line 25) <https://reviews.apache.org/r/50965/#comment211629> if this is a base class for HAWQ tests, maybe call it HAWQTestCase to denote that this is not a test class by itself ? ambari-server/src/test/python/common-services/HAWQ/test_hawqcommon.py (line 27) <https://reviews.apache.org/r/50965/#comment211630> this is where packages for HAWQ are, so COMMON_SERVICES_PACKAGE_DIR is a bit too general name for HAWQ specific directory ambari-server/src/test/python/common-services/PXF/test_alerts_api_status.py (line 28) <https://reviews.apache.org/r/50965/#comment211631> PXF_ALERTS_DIR_PATH would be a better name - Alexander Denissov On Aug. 10, 2016, 8:30 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50965/ > --- > > (Updated Aug. 10, 2016, 8:30 p.m.) > > > Review request for Ambari, Alexander Denissov and Matt. > > > Bugs: AMBARI-18110 > https://issues.apache.org/jira/browse/AMBARI-18110 > > > Repository: ambari > > > Description > --- > > Ambari build jobs are not running the unit tests for HAWQ. > HAWQ (and PXF) unit tests in the new common-services directory under test is > being ignored during mvn's test phase. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 529c492 > > ambari-server/src/test/python/common-services/HAWQ/test_alert_component_status.py > b2e1d4d > > ambari-server/src/test/python/common-services/HAWQ/test_alert_segment_registration_status.py > 4066e29 > > ambari-server/src/test/python/common-services/HAWQ/test_alert_sync_status.py > a44cad1 > ambari-server/src/test/python/common-services/HAWQ/test_hawqcommon.py > PRE-CREATION > ambari-server/src/test/python/common-services/HAWQ/test_hawqmaster.py > 62d54cf > ambari-server/src/test/python/common-services/HAWQ/test_hawqsegment.py > 48203ca > ambari-server/src/test/python/common-services/HAWQ/test_hawqstandby.py > 8674549 > ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py > e0f1c43 > ambari-server/src/test/python/common-services/PXF/test_alerts_api_status.py > ee187e2 > ambari-server/src/test/python/common-services/PXF/test_service_advisor.py > 8b2a4ba > ambari-server/src/test/python/unitTests.py b010804 > > Diff: https://reviews.apache.org/r/50965/diff/ > > > Testing > --- > > ``` > Running tests for stack:None service:HAWQ > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSync
Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50731/#review144720 --- Ship it! Ship It! - Alexander Denissov On Aug. 4, 2016, 1:11 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50731/ > --- > > (Updated Aug. 4, 2016, 1:11 a.m.) > > > Review request for Ambari, Alexander Denissov and Lav Jain. > > > Bugs: AMBARI-17996 > https://issues.apache.org/jira/browse/AMBARI-17996 > > > Repository: ambari > > > Description > --- > > HAWQ service advisor shows wrong recommendations on edge cases > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 9baaa1c > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > 0777dc6 > ambari-server/src/main/resources/stacks/stack_advisor.py 437fe4f > ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py > 7ececa0 > > Diff: https://reviews.apache.org/r/50731/diff/ > > > Testing > --- > > Manually tested on live cluster. > > Added unit tests > ``` > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Remove HAWQ Standby ... 2016-08-02 18:01:23,293 - Removing > HAWQ Standby Master ... > ok > test_resync_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Resync HAWQ Standby ... 2016-08-02 18:01:23,295 - HAWQ > Standby Master Re-Sync started in fast mode... > ok > test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not > Kerberized ... 2016-08-02 18:01:23,298 - Executing HAWQ Check ... > ok > test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, > Kerberized ... 2016-08-02 18:01:23,304 - Executing HAWQ Check ... > ok > test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not > Kerberized ... 2016-08-02 18:01:23,309 - Executing HAWQ Check ... > ok >
Review Request 50777: AMBARI-18014: PXF service definition is missing pxf-json profile
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50777/ --- Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and Matt. Bugs: AMBARI-18014 https://issues.apache.org/jira/browse/AMBARI-18014 Repository: ambari Description --- PXF service definition is missing pxf-json profile Diffs - ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-profiles.xml 69e3228 ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 5978d0b ambari-server/src/test/python/common-services/PXF/test_pxf.py 49df75d ambari-server/src/test/python/common-services/configs/pxf_default.json 10cd9a4 Diff: https://reviews.apache.org/r/50777/diff/ Testing --- -- Ran 16 tests in 0.050s OK Thanks, Alexander Denissov
Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50731/#review144642
---
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
(line 167)
<https://reviews.apache.org/r/50731/#comment210670>
move these after if statement ?
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
(line 173)
<https://reviews.apache.org/r/50731/#comment210669>
probably need to analyze if the changed configs are only for HAWQ service
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
(line 237)
<https://reviews.apache.org/r/50731/#comment210675>
might be an edge case arouns YARN RM-HA properties
ambari-server/src/main/resources/stacks/stack_advisor.py (line 900)
<https://reviews.apache.org/r/50731/#comment210676>
in names.keys() ?
- Alexander Denissov
On Aug. 3, 2016, 1:01 a.m., Matt wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50731/
> ---
>
> (Updated Aug. 3, 2016, 1:01 a.m.)
>
>
> Review request for Ambari, Alexander Denissov and Lav Jain.
>
>
> Bugs: AMBARI-17996
> https://issues.apache.org/jira/browse/AMBARI-17996
>
>
> Repository: ambari
>
>
> Description
> ---
>
> HAWQ service advisor shows wrong recommendations on edge cases
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
> 9baaa1c
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
> 0777dc6
> ambari-server/src/main/resources/stacks/stack_advisor.py 437fe4f
> ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py
> 7ececa0
>
> Diff: https://reviews.apache.org/r/50731/diff/
>
>
> Testing
> ---
>
> Manually tested on live cluster.
>
> Added unit tests
> ```
> test_hawq_master_critical
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus)
> ... ok
> test_hawq_segment_critical
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus)
> ... ok
> test_hawq_standby_critical
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus)
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus)
> ... ok
> test_exception_registration_status
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ...
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-08-02 18:01:23,293 - Removing
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-08-
Re: Review Request 50547: AMBARI-17939: Validations are missing for HDFS properties recommended by HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50547/ --- (Updated July 28, 2016, 9:30 p.m.) Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and Matt. Changes --- updated patch, removed HAWQ recommendations / validations from HDP 2.3 stack_advisor, as they are now available in HAWQServiceAdvisor Bugs: AMBARI-17939 https://issues.apache.org/jira/browse/AMBARI-17939 Repository: ambari Description --- When HAWQ is added to existing cluster, some HDFS properties do not take recommended values, so there should be validations that alert the user to the fact that the properties need to be updated for HAWQ to function optimally. Diffs (updated) - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 9f85518 ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py f82655d ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py b3edae4 ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py ff87adb Diff: https://reviews.apache.org/r/50547/diff/ Testing --- tested manually and update unit tests: -- Ran 66 tests in 0.285s OK Thanks, Alexander Denissov
Review Request 50547: AMBARI-17939: Validations are missing for HDFS properties recommended by HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50547/ --- Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and Matt. Bugs: AMBARI-17939 https://issues.apache.org/jira/browse/AMBARI-17939 Repository: ambari Description --- When HAWQ is added to existing cluster, some HDFS properties do not take recommended values, so there should be validations that alert the user to the fact that the properties need to be updated for HAWQ to function optimally. Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 9f85518 ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py b3edae4 Diff: https://reviews.apache.org/r/50547/diff/ Testing --- tested manually and update unit tests: -- Ran 66 tests in 0.285s OK Thanks, Alexander Denissov
Re: Review Request 50544: Ambari install/init should create a new gpadmin database
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50544/#review143868 --- Ship it! Ship It! - Alexander Denissov On July 27, 2016, 11:41 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50544/ > --- > > (Updated July 27, 2016, 11:41 p.m.) > > > Review request for Ambari, Alexander Denissov and Matt. > > > Bugs: AMBARI-17937 > https://issues.apache.org/jira/browse/AMBARI-17937 > > > Repository: ambari > > > Description > --- > > If you are logged in as gpadmin on the master and type in "psql" to connect > to the database, it will fail. psql assumes you want to connect to the > database named "gpadmin" which matches your username. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 8c43555 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py > 55a5982 > > Diff: https://reviews.apache.org/r/50544/diff/ > > > Testing > --- > > Tested manually. > > > Thanks, > > Lav Jain > >
Re: Review Request 50198: HAWQ fails to execute queries in low-memory environment
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50198/#review142834 --- Ship it! Ship It! - Alexander Denissov On July 19, 2016, 7:15 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50198/ > --- > > (Updated July 19, 2016, 7:15 p.m.) > > > Review request for Ambari, Alexander Denissov and Lav Jain. > > > Bugs: AMBARI-17720 > https://issues.apache.org/jira/browse/AMBARI-17720 > > > Repository: ambari > > > Description > --- > > When a user is running on vagrant or single node with low system memory, > configurations are not sufficient to allocate containers for "default" 6 > vsegs per node especially during HASH dist table query or external table > query. > > Stack advisor should adjust the config values to ensure service checks and > queries (including Random tables, HASH dist tables, PXF ext tables) do not > fail. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 0bddb07 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > dffe57d > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > 4089b75 > ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py > f8a9468 > ambari-server/src/test/python/common-services/PXF/test_service_advisor.py > 7510e5f > > Diff: https://reviews.apache.org/r/50198/diff/ > > > Testing > --- > > Tested on Live cluster > > Added unit tests: > ``` > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Remove HAWQ Standby ... 2016-07-19 12:12:33,657 - Removing > HAWQ Standby Master ... > ok > test_resync_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Resync HAWQ Standby ... 2016-07-19 12:12:33,660 - HAWQ > Standby Master Re-Sync started in fast mode... > ok > test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not > Kerberized ... 2016-07-19 12:12:33,663 - Exec
Re: Review Request 50198: HAWQ fails to execute queries in low-memory environment
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50198/#review142831 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 213) <https://reviews.apache.org/r/50198/#comment208504> this property has no impact on the logic inside this if, maybe move it out of the if statement ? - Alexander Denissov On July 19, 2016, 7:15 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50198/ > --- > > (Updated July 19, 2016, 7:15 p.m.) > > > Review request for Ambari, Alexander Denissov and Lav Jain. > > > Bugs: AMBARI-17720 > https://issues.apache.org/jira/browse/AMBARI-17720 > > > Repository: ambari > > > Description > --- > > When a user is running on vagrant or single node with low system memory, > configurations are not sufficient to allocate containers for "default" 6 > vsegs per node especially during HASH dist table query or external table > query. > > Stack advisor should adjust the config values to ensure service checks and > queries (including Random tables, HASH dist tables, PXF ext tables) do not > fail. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 0bddb07 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > dffe57d > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > 4089b75 > ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py > f8a9468 > ambari-server/src/test/python/common-services/PXF/test_service_advisor.py > 7510e5f > > Diff: https://reviews.apache.org/r/50198/diff/ > > > Testing > --- > > Tested on Live cluster > > Added unit tests: > ``` > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Remove HAWQ Standby ... 2016-07-19 12:12:33,657 - Removing > HAWQ Standby Master ... > ok > test_resync_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Resync HAWQ Standby ... 2016-07-19 12:12:33,660 - HAWQ > Standby Master Re
Re: Review Request 50097: HAWQ install should recommend required values for HDFS properties
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50097/ --- (Updated July 18, 2016, 10:01 p.m.) Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and Matt. Changes --- rebased against latest Bugs: AMBARI-17719 https://issues.apache.org/jira/browse/AMBARI-17719 Repository: ambari Description --- HAWQ install should recommend required values for HDFS properties Diffs (updated) - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py feb3d52 ambari-server/src/main/resources/stacks/stack_advisor.py fb30f4d ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 780370d Diff: https://reviews.apache.org/r/50097/diff/ Testing --- Tested manually, updated unit tests. Thanks, Alexander Denissov
Review Request 50097: HAWQ install should recommend required values for HDFS properties
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50097/ --- Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and Matt. Bugs: AMBARI-17719 https://issues.apache.org/jira/browse/AMBARI-17719 Repository: ambari Description --- HAWQ install should recommend required values for HDFS properties Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 16e2952 ambari-server/src/main/resources/stacks/stack_advisor.py 47f4500 ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 780370d Diff: https://reviews.apache.org/r/50097/diff/ Testing --- Tested manually, updated unit tests. Thanks, Alexander Denissov
Re: Review Request 50086: Slider range for should be dynamic on HAWQ settings page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50086/#review142449 --- Ship it! Ship It! - Alexander Denissov On July 15, 2016, 9:24 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50086/ > --- > > (Updated July 15, 2016, 9:24 p.m.) > > > Review request for Ambari, Alexander Denissov, Jayush Luniya, Matt, and Tim > Thorpe. > > > Bugs: AMBARI-17700 > https://issues.apache.org/jira/browse/AMBARI-17700 > > > Repository: ambari > > > Description > --- > > Ambari should dynamically update the slider range on HAWQ parameters on > settings page depending on the cluster size because the static range isn't > usable and does not allow selecting a smaller desired value. E.g. the maximum > value of 1 is too large when setting it to less than 10. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 150b2c6 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > 16e2952 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json > 3c8a897 > > Diff: https://reviews.apache.org/r/50086/diff/ > > > Testing > --- > > Manually tested > > > Thanks, > > Lav Jain > >
Re: Review Request 49362: Add additional parameters to hawq-site.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49362/#review140036 --- Ship it! Ship It! - Alexander Denissov On June 29, 2016, 10:04 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49362/ > --- > > (Updated June 29, 2016, 10:04 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-17473 > https://issues.apache.org/jira/browse/AMBARI-17473 > > > Repository: ambari > > > Description > --- > > Add additional parameters to hawq-site.xml > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 6e21d84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml > 1d27052 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json > d7940a1 > > Diff: https://reviews.apache.org/r/49362/diff/ > > > Testing > --- > > Manually tested. > > > Thanks, > > Matt > >
Re: Review Request 48290: Change HAWQ DFS Url from hawq_default to hawq_data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48290/#review136358 --- Ship it! Ship It! - Alexander Denissov On June 6, 2016, 7:17 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48290/ > --- > > (Updated June 6, 2016, 7:17 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-17070 > https://issues.apache.org/jira/browse/AMBARI-17070 > > > Repository: ambari > > > Description > --- > > Change HAWQ DFS Url from hawq_default to hawq_data > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 3ac4e89 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py > a8a4487 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > ec04fd2 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 70d7ed1 > ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json b839de6 > ambari-web/app/data/HDP2/ha_properties.js 18c5a68 > > ambari-web/test/controllers/main/admin/highAvailability/nameNode/step3_controller_test.js > ca55a32 > ambari-web/test/utils/configs/config_initializer_test.js 48eae23 > > Diff: https://reviews.apache.org/r/48290/diff/ > > > Testing > --- > > Waiting for results from Hadoop QA > > > Thanks, > > Matt > >
Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48235/#review136128 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 182) <https://reviews.apache.org/r/48235/#comment201140> better to move these lines higher to after line 149 where the similar logic resides ? ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 190) <https://reviews.apache.org/r/48235/#comment201141> are we repeating default values already defined in .xml file ? Would be better not to do that, if possible. - Alexander Denissov On June 3, 2016, 10:49 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48235/ > --- > > (Updated June 3, 2016, 10:49 p.m.) > > > Review request for Ambari, Alexander Denissov, Lav Jain, and Matt. > > > Bugs: AMBARI-17023 > https://issues.apache.org/jira/browse/AMBARI-17023 > > > Repository: ambari > > > Description > --- > > Show only relevant properties based on the status of HAWQ Resource Manager. > When Yarn mode is enable, only two parameters should be invisible. > hawq_rm_memory_limit_perseg > hawq_rm_nvcore_limit_perseg > When standalone mode is enabled, four parameters should be invisible. what > you list are correct. > hawq_rm_yarn_app_name > hawq_rm_yarn_queue_name > hawq_rm_yarn_scheduler_address > hawq_rm_yarn_address > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 3ac4e89 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > 28eb82f > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > 5750938 > > Diff: https://reviews.apache.org/r/48235/diff/ > > > Testing > --- > > yes. > Now testing > /Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ > ~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ > ~/github/ambari-toolbox/vagrant/ambari > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster
Re: Review Request 47802: expose vm.overcommit_ratio on hawq-sysctl-env
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47802/#review134692 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 207) <https://reviews.apache.org/r/47802/#comment199567> if user manually puts a value for this field, will we override it with our recommendations every time they navigate to the config page ? ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json (line 201) <https://reviews.apache.org/r/47802/#comment199566> can this be a slider between 0 ad 100 ? Should we also have validation that the value is within that range if user manually enters the number ? - Alexander Denissov On May 25, 2016, 12:17 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47802/ > --- > > (Updated May 25, 2016, 12:17 a.m.) > > > Review request for Ambari, Alexander Denissov, Lav Jain, and Matt. > > > Bugs: AMBARI-16827 > https://issues.apache.org/jira/browse/AMBARI-16827 > > > Repository: ambari > > > Description > --- > > expose vm.overcommit_ratio on hawq-sysctl-env > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml > c51b012 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > 836144c > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json > e4d1c45 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > 53c5574 > > Diff: https://reviews.apache.org/r/47802/diff/ > > > Testing > --- > > yes. test case added. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47749: Warning for HAWQSTANDBY does not show up on Assign Masters page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47749/#review134514 --- Ship it! Ship It! - Alexander Denissov On May 23, 2016, 11:44 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47749/ > --- > > (Updated May 23, 2016, 11:44 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-16829 > https://issues.apache.org/jira/browse/AMBARI-16829 > > > Repository: ambari > > > Description > --- > > Warning for HAWQSTANDBY does not show up on Assign Masters page > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > c5813ad > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > c33a223 > > Diff: https://reviews.apache.org/r/47749/diff/ > > > Testing > --- > > Manually tested. > > Added new test case. > > ``` > test_createComponentLayoutRecommendations_hawq_1_Host > (test_stack_advisor.TestHDP23StackAdvisor) ... ok > test_createComponentLayoutRecommendations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT does not get recommended during Add Service Wizard, > when HAWQ has already been installed ... ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, > when HAWQ is selected for installation ... ok > test_createComponentLayoutRecommendations_hawqsegment_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Cluster Install > Wizard, when HAWQ is selected for installation ... ok > test_createComponentLayoutRecommendations_no_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF does not get recommended during Add Service Wizard, when PXF > has already been installed ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF gets recommended correctly during Add Service Wizard, when PXF > is selected for installation ... ok > test_createComponentLayoutRecommendations_pxf_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF gets recommended correctly during Cluster Install Wizard, when > PXF is selected for installation ... ok > test_getComponentLayoutValidations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test layout validations for HAWQ components on a 3-node cluster ... ok > test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode > (test_stack_advisor.TestHDP23StackAdvisor) > Test validation warning for HAWQ segment not colocated with DATANODE ... ok > test_getComponentLayoutValidations_nohawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components on a 3-node cluster ... ok > test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn > (test_stack_advisor.TestHDP23StackAdvisor) > Test NO warning is generated when PXF is co-located with NAMENODE and > DATANODE ... ok > test_getComponentLayoutValidations_pxf_not_co_located_with_dn > (test_stack_advisor.TestHDP23StackAdvisor) > Test warning is generated when PXF is not co-located with NAMENODE or > DATANODE ... ok > test_getComponentLayoutValidations_pxf_not_co_located_with_nn > (test_stack_advisor.TestHDP23StackAdvisor) > Test warning is generated when PXF is not co-located with NAMENODE ... ok > test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn > (test_stack_advisor.TestHDP23StackAdvisor) > Test warning is generated when PXF is not co-located with NAMENODE or > DATANODE ... ok > test_getComponentLayoutValidations_sparkts_no_hive > (test_stack_advisor.TestHDP23StackAdvisor) > Test SparkTS is picked when Hive is not installed ..
Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47656/#review134330 --- ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java (line 51) <https://reviews.apache.org/r/47656/#comment199136> are these supposed to be commented out ? if yes, maybe remove them ? ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java (line 305) <https://reviews.apache.org/r/47656/#comment199138> a lot of commented out code here - Alexander Denissov On May 20, 2016, 7:03 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47656/ > --- > > (Updated May 20, 2016, 7:03 p.m.) > > > Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh > chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth > Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-12885 > https://issues.apache.org/jira/browse/AMBARI-12885 > > > Repository: ambari > > > Description > --- > > The purpose of this proposal is to facilitate adding custom services to an > existing stack. Ideally this would support adding and upgrading custom > services separately from the core services defined in the stack. In > particular we are looking at custom services that need to support several > different stacks (different distributions of Ambari). The release cycle of > the custom services may be different from that of the core stack; that is, a > custom service may be upgraded at a different rate than the core distribution > itself and may be upgraded multiple times within the lifespan of a single > release of the core distribution. > > One possible approach to handling this would be dynamically extending a stack > (after install time). It would be best to extend the stack in packages where > a stack extension package can have one or more custom services. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py > fcea23f > ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee > ambari-server/conf/unix/ambari.properties 9f1692e > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java > 9c864b6 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > c54fe3f > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java > 557ce98 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > f5e8f39 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > d6b9d0e > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > dd3d098 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java > 3721113 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java > PRE-CREATION > &
Re: Review Request 47642: Add HAWQ Standby wizard should recommend HAWQSTANDBY correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47642/#review134328 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 52) <https://reviews.apache.org/r/47642/#comment199134> what is standby was also already assigned and is at position 0 in hawqMasterHosts[] ? The the logic will break apart, or the whole method will not even be called if standby is assigned ? ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 55) <https://reviews.apache.org/r/47642/#comment199135> why just not say availableHosts[0] as it is the same as availableHosts[:1] but less awkward ? - Alexander Denissov On May 21, 2016, 12:07 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47642/ > --- > > (Updated May 21, 2016, 12:07 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-16790 > https://issues.apache.org/jira/browse/AMBARI-16790 > > > Repository: ambari > > > Description > --- > > On assign masters page of Add HAWQ Standby wizard, a best effort should be > made such that HAWQSTANDBY is not co-located with HAWQMASTER or AMBARI_SERVER. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > 0590604 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > 905d455 > > Diff: https://reviews.apache.org/r/47642/diff/ > > > Testing > --- > > Manually tested. > > Added unit tests: > ``` > test_createComponentLayoutRecommendations_hawq_1_Host > (test_stack_advisor.TestHDP23StackAdvisor) ... ok > test_createComponentLayoutRecommendations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT does not get recommended during Add Service Wizard, > when HAWQ has already been installed ... ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, > when HAWQ is selected for installation ... ok > test_createComponentLayoutRecommendations_hawqsegment_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Cluster Install > Wizard, when HAWQ is selected for installation ... ok > test_createComponentLayoutRecommendations_no_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF does not get recommended during Add Service Wizard, when PXF > has already been installed ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF gets recommended correctly during Add Service Wizard, when PXF > is selected for installation ... ok > test_createComponentLayoutRecommendations_pxf_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF gets recommended correctly during Cluster Install Wizard, when > PXF is selected for installation ... ok > test_getComponentLayoutValidations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test layout validations for HAWQ components on a 3-node cluster ... ok > test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode > (test_stack_advisor.TestHDP23StackAdvisor) > Test validation warning for HAWQ segment not colocated with DATANODE ... ok > test_getComponentLayoutValidations_nohawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components on a 3-node cluster ... ok > test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn > (test_stack_advisor.TestHDP23StackAdvisor) > Test NO warning is generated when PXF is co-located with NAMENODE and > DATANODE ... ok > test_getComponentLayoutValidations_pxf_not_co_located_with_dn > (test_stack_advisor.TestHDP2
Re: Review Request 47627: hawq_standby_address_host property should be removed from configuration if HAWQSTANDBY component does not exist in BP
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47627/#review134325 --- ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java (line 3086) <https://reviews.apache.org/r/47627/#comment199130> Since this method is called for every property, we should optimize the performance by reaching the decision faster in 99% of cases where the conditions do not apply. If setOfHawqPropertyNamesNonHA.contains(propertyName) operation is less expensive (O(1) for the set) then topology.getHostGroupsForComponent(HAWQSTANDBY).size() (which might search for component inside each host group), then maybe contains() operation should be the first check in the if condition. Also, merging 2 ifs into 1 with && will be more readable. - Alexander Denissov On May 20, 2016, 10:11 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47627/ > --- > > (Updated May 20, 2016, 10:11 p.m.) > > > Review request for Ambari, Alexander Denissov, Matt, and Robert Nettleton. > > > Bugs: AMBARI-16786 > https://issues.apache.org/jira/browse/AMBARI-16786 > > > Repository: ambari > > > Description > --- > > hawq_standby_address_host property should be removed from configuration if > HAWQSTANDBY component does not exist in BP > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 1ddc4e1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 5c6aeaf > > Diff: https://reviews.apache.org/r/47627/diff/ > > > Testing > --- > > yes. test case added. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47580: Add pxf-hbase.jar to HBASE_CLASSPATH through service_advisor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47580/#review133977 --- ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py (line 74) <https://reviews.apache.org/r/47580/#comment198648> I would check only for presence of pxf-hbase.jar string, in case users edit the content with extra information ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py (line 84) <https://reviews.apache.org/r/47580/#comment198659> here as well, just check for the jar ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py (line 86) <https://reviews.apache.org/r/47580/#comment198660> pxf-hbase.jar -- h is missing - Alexander Denissov On May 19, 2016, 6:11 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47580/ > --- > > (Updated May 19, 2016, 6:11 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-16752 > https://issues.apache.org/jira/browse/AMBARI-16752 > > > Repository: ambari > > > Description > --- > > In service install wizard, pxf-hbase.jar should be added to the > HBASE_CLASSPATH (recommendation and validation) > > > Diffs > - > > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > e2510eb > ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py > PRE-CREATION > > Diff: https://reviews.apache.org/r/47580/diff/ > > > Testing > --- > > Manually Tested. > > Unit tests added. > > Hadoop QA: > ``` > +1 overall. Here are the results of testing the latest attachment > http://issues.apache.org/jira/secure/attachment/12804840/AMBARI-16752-trunk-orig.patch > against trunk revision . > +1 @author. The patch does not contain any @author tags. > +1 tests included. The patch appears to include 1 new or modified test files. > +1 javac. The applied patch does not increase the total number of javac > compiler warnings. > +1 release audit. The applied patch does not increase the total number of > release audit warnings. > +1 core tests. The patch passed unit tests in ambari-server. > Test results: > https://builds.apache.org/job/Ambari-trunk-test-patch/6889//testReport/ > Console output: > https://builds.apache.org/job/Ambari-trunk-test-patch/6889//console > This message is automatically generated. > ``` > > > Thanks, > > Matt > >
Re: Review Request 47573: Next button is disabled on assign master page in Add Hawq Standby wizard
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47573/#review133872 --- ambari-web/app/views/main/admin/highAvailability/hawq/addStandby/wizard_view.js (line 64) <https://reviews.apache.org/r/47573/#comment198536> does this single fix solves both issues ? - Alexander Denissov On May 19, 2016, 1:29 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47573/ > --- > > (Updated May 19, 2016, 1:29 a.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16754 > https://issues.apache.org/jira/browse/AMBARI-16754 > > > Repository: ambari > > > Description > --- > > Next button is disabled on assign master page in Add Hawq Standby wizard. > Host maintenance state is not propageted from the wizard view to the > underlying controllers. > > > Diffs > - > > > ambari-web/app/views/main/admin/highAvailability/hawq/addStandby/wizard_view.js > def93f3 > > Diff: https://reviews.apache.org/r/47573/diff/ > > > Testing > --- > > yes. > > > 27821 tests complete (31 seconds) > 154 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 01:32 min > [INFO] Finished at: 2016-05-18T18:26:48-07:00 > [INFO] Final Memory: 14M/339M > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47434: Remove unused parameters from hawq-site.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47434/#review133471 --- Ship it! Ship It! - Alexander Denissov On May 17, 2016, 12:14 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47434/ > --- > > (Updated May 17, 2016, 12:14 a.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16694 > https://issues.apache.org/jira/browse/AMBARI-16694 > > > Repository: ambari > > > Description > --- > > Remove the below unused parameters from hawq-site.xml > hawq_re_cgroup_hierarchy_name > hawq_re_cgroup_mount_point > hawq_re_cpu_enable > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 170e8cf > > Diff: https://reviews.apache.org/r/47434/diff/ > > > Testing > --- > > yes. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47328/#review133154 --- Ship it! Ship It! - Alexander Denissov On May 13, 2016, 12:58 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47328/ > --- > > (Updated May 13, 2016, 12:58 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16646 > https://issues.apache.org/jira/browse/AMBARI-16646 > > > Repository: ambari > > > Description > --- > > Consider all HAWQ host machines. > If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory > as 2, else use 1 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > a26a398 > ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > PRE-CREATION > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > be0f3e4 > > Diff: https://reviews.apache.org/r/47328/diff/ > > > Testing > --- > > Added unit tests. > > ``` > bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/ > bash-3.2$ python -m discover -v > test_createComponentLayoutRecommendations_hawq_1_Host > (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation > for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor > implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT does not get recommended during Add Service Wizard, > when HAWQ has already been installed ... ServiceAdvisor implementation for > service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, > when HAWQ is selected for installation ... ServiceAdvisor implementation for > service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Cluster Install > Wizard, when HAWQ is selected for installation ... ServiceAdvisor > implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_no_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_insta
Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47328/#review133146 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 166) <https://reviews.apache.org/r/47328/#comment197443> fancy, but this is a constant that we don;t have to calculate every time, why not just say MEM_THRESHOLD = 33554432 # 32GB in KB ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py (line 2041) <https://reviews.apache.org/r/47328/#comment197444> should HAWQ/PXF tests be now moved to separate service advisor tests files, perhaps as a separate effort ? - Alexander Denissov On May 13, 2016, 12:58 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47328/ > --- > > (Updated May 13, 2016, 12:58 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16646 > https://issues.apache.org/jira/browse/AMBARI-16646 > > > Repository: ambari > > > Description > --- > > Consider all HAWQ host machines. > If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory > as 2, else use 1 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > a26a398 > ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > PRE-CREATION > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > be0f3e4 > > Diff: https://reviews.apache.org/r/47328/diff/ > > > Testing > --- > > Added unit tests. > > ``` > bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/ > bash-3.2$ python -m discover -v > test_createComponentLayoutRecommendations_hawq_1_Host > (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation > for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor > implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT does not get recommended during Add Service Wizard, > when HAWQ has already been installed ... ServiceAdvisor implementation for > service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, > when HAWQ is selected for installation ... ServiceAdvisor implementation for > service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Cluster I
Re: Review Request 47252: PXF Agents Live Widget on Dashboard needs custom widget to display only single threshold i.e., Green/Red only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47252/#review132943 --- ambari-web/app/messages.js (line 2659) <https://reviews.apache.org/r/47252/#comment197205> a proper grammar is "between X and Y", I believe, not "between X to Y" ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js (line 31) <https://reviews.apache.org/r/47252/#comment197206> if the threshold is single, then we don't need 1 at the end of variable name. ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js (line 111) <https://reviews.apache.org/r/47252/#comment197209> we don't have orange here, do we ? - Alexander Denissov On May 11, 2016, 7:04 p.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47252/ > --- > > (Updated May 11, 2016, 7:04 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, > Lav Jain, and Matt. > > > Bugs: AMBARI-16461 > https://issues.apache.org/jira/browse/AMBARI-16461 > > > Repository: ambari > > > Description > --- > > PXF Agents Live Widget on Dashboard needs custom widget to display only > single threshold i.e., Green/Red only > > > Diffs > - > > ambari-web/app/messages.js c5e1145 > ambari-web/app/mixins.js cf8cef6 > ambari-web/app/mixins/main/dashboard/widgets/editable_with_limit.js 97b6330 > ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js > PRE-CREATION > ambari-web/app/styles/modal_popups.less 5e89dbc > > ambari-web/app/templates/main/dashboard/edit_widget_popup_single_threshold.hbs > PRE-CREATION > ambari-web/app/views.js 127a996 > ambari-web/app/views/main/dashboard/widgets.js 38181c0 > ambari-web/app/views/main/dashboard/widgets/pxf_live.js 75f5886 > ambari-web/app/views/main/dashboard/widgets/text_widget_single_threshold.js > PRE-CREATION > > Diff: https://reviews.apache.org/r/47252/diff/ > > > Testing > --- > > Yes, All tests passed. Test was already in place. > > > Thanks, > > Goutam Tadi > >
Re: Review Request 47134: HAWQ Password handling updated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47134/#review132347 --- Ship it! Ship It! - Alexander Denissov On May 9, 2016, 7:55 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47134/ > --- > > (Updated May 9, 2016, 7:55 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16386 > https://issues.apache.org/jira/browse/AMBARI-16386 > > > Repository: ambari > > > Description > --- > > Updated salt used while creating HAWQ user, and also ensured that the > password is treated exactly as entered on the HAWQ config ui. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 1bdc85a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 670a58b > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 2671202 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 7963b3d > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 605f49a > > Diff: https://reviews.apache.org/r/47134/diff/ > > > Testing > --- > > yes. > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Remove HAWQ Standby ... 2016-05-09 12:54:41,958 - Removing > HAWQ Standby Master ... > ok > test_resync_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Resync HAWQ Standby ... 2016-05-09 12:54:41,963 - HAWQ > Standby Master Re-Sync started in fast mode... > ok > test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not > Kerberized ... 2016-05-09 12:54:41,968 - Executing HAWQ Check ... > ok > test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, > Kerberized ... 2016-05-09 12:54:41,978 - Executing HAWQ Check ... > ok > test_run_hawq_check_case11 (tes
Review Request 46996: HAWQ service can be stopped and checked even when STANDBY MASTER is stopped
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46996/ --- Review request for Ambari, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Lav Jain, and Matt. Bugs: AMBARI-16236 https://issues.apache.org/jira/browse/AMBARI-16236 Repository: ambari Description --- HAWQ service can be stopped and checked even when STANDBY MASTER is stopped Diffs - ambari-web/app/controllers/main/service/item.js bfede8a ambari-web/test/controllers/main/service/item_test.js 4bac784 Diff: https://reviews.apache.org/r/46996/diff/ Testing --- manual. added unit tests: 27532 tests complete (29 seconds) 154 tests pending [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 45.229 s [INFO] Finished at: 2016-05-04T16:25:14-08:00 Thanks, Alexander Denissov
Re: Review Request 46949: PXF alert: change the message to make it more meaningful when both namenodes are down on Secured HA cluster.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46949/#review131752 --- Ship it! Ship It! - Alexander Denissov On May 4, 2016, 9:28 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46949/ > --- > > (Updated May 4, 2016, 9:28 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-16237 > https://issues.apache.org/jira/browse/AMBARI-16237 > > > Repository: ambari > > > Description > --- > > PXF alerts shows "Configuration parameter 'fs_root' was not found in > configurations dictionary!" when both namnodes are down on secured NN HA > environment, which the error message is not too clear. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py > ee16c9b > > Diff: https://reviews.apache.org/r/46949/diff/ > > > Testing > --- > > Manually tested. > > > Thanks, > > jun aoki > >
Re: Review Request 46949: PXF alert: change the message to make it more meaningful when both namenodes are down on Secured HA cluster.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46949/#review131705
---
ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py
(line 142)
<https://reviews.apache.org/r/46949/#comment195735>
The code in line 57 is:
raise Fail('No active NameNode was found.')
Should the same be done here for consistency ?
- Alexander Denissov
On May 3, 2016, 11:05 p.m., jun aoki wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46949/
> ---
>
> (Updated May 3, 2016, 11:05 p.m.)
>
>
> Review request for Ambari.
>
>
> Bugs: AMBARI-16237
> https://issues.apache.org/jira/browse/AMBARI-16237
>
>
> Repository: ambari
>
>
> Description
> ---
>
> PXF alerts shows "Configuration parameter 'fs_root' was not found in
> configurations dictionary!" when both namnodes are down on secured NN HA
> environment, which the error message is not too clear.
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py
> ee16c9b
>
> Diff: https://reviews.apache.org/r/46949/diff/
>
>
> Testing
> ---
>
> Manually tested.
>
>
> Thanks,
>
> jun aoki
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On April 29, 2016, 12:12 a.m., Alexander Denissov wrote: > > Do we have a branch cut with these changes in so that we can test HAWQ and > > PXF with this new logic ? After testing the branch can be merged to trunk > > and we will avoid any major surprises. > > Tim Thorpe wrote: > There isn't currently a branch. Do we really need a branch, wouldn't it > be just as easy to just apply the patch over trunk and try it out? If you > think we need a branch then, I'll ask that one get created. > > Alexander Denissov wrote: > We will be testing this patch with HAWQ to see if it behaves as expected > and there are no regressions. > > Tim Thorpe wrote: > Hi Alex, please let me know when you have validated the patch with HAWQ > so I can get it pushed in (after I try it out again over trunk). Thanks Tim, the guys tried it on Friday and it did not quite work (custom recommendations did not come in UI). They are trying to dig deeper and debug today to pinpoint where the problem is. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131020 --- On April 29, 2016, 8:11 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 29, 2016, 8:11 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/properties.json eac0dbd > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > f475798 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 2080c52 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 46852: HAWQ Configuration should be updated whenever Namenode is being moved using the move namenode wizard
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46852/#review131348 --- Ship it! Ship It! - Alexander Denissov On April 29, 2016, 8:50 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46852/ > --- > > (Updated April 29, 2016, 8:50 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16191 > https://issues.apache.org/jira/browse/AMBARI-16191 > > > Repository: ambari > > > Description > --- > > HAWQ Configuration should be updated whenever Namenode is being moved using > the move namenode wizard. > Parameters which should be updated > Case 1: HDFS HA > hdfs-client parameters under HAWQ service > dfs.namenode.rpc-address.. > dfs.namenode.http-address.. > Case 2: Non HA HDFS > hawq-site parmeters under HAWQ service > hawq_dfs_url > > > Diffs > - > > ambari-web/app/controllers/main/service/reassign/step4_controller.js > a20aa82 > ambari-web/app/utils/configs/move_namenode_config_initializer.js 160848c > ambari-web/test/controllers/main/service/reassign/step4_controller_test.js > 460726b > > Diff: https://reviews.apache.org/r/46852/diff/ > > > Testing > --- > > yes. > 27472 tests complete (29 seconds) > 154 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 46.556 s > [INFO] Finished at: 2016-04-29T13:53:51-07:00 > [INFO] Final Memory: 14M/338M > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On April 29, 2016, 12:12 a.m., Alexander Denissov wrote: > > Do we have a branch cut with these changes in so that we can test HAWQ and > > PXF with this new logic ? After testing the branch can be merged to trunk > > and we will avoid any major surprises. > > Tim Thorpe wrote: > There isn't currently a branch. Do we really need a branch, wouldn't it > be just as easy to just apply the patch over trunk and try it out? If you > think we need a branch then, I'll ask that one get created. We will be testing this patch with HAWQ to see if it behaves as expected and there are no regressions. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131020 --- On April 29, 2016, 8:11 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 29, 2016, 8:11 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/properties.json eac0dbd > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > f475798 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 2080c52 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 46800: HAWQ Configuration should be updated whenever Namenode is being moved.
> On April 29, 2016, 8:59 p.m., Alexander Denissov wrote: > > Ship It! Do we also need to move PXF to the new namenode host ? - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46800/#review131181 --- On April 29, 2016, 8:46 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46800/ > --- > > (Updated April 29, 2016, 8:46 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, Oleksandr > Diachenko, and Richard Zang. > > > Bugs: AMBARI-16172 > https://issues.apache.org/jira/browse/AMBARI-16172 > > > Repository: ambari > > > Description > --- > > HAWQ Configuration should be updated whenever Namenode is being moved using > the move namenode wizard. > Parameters which should be updated > Case 1: HDFS HA > hdfs-client parameters under HAWQ service > dfs.namenode.rpc-address.. > dfs.namenode.http-address.. > Case 2: Non HA HDFS > hawq-site parmeters under HAWQ service > hawq_dfs_url > > > Diffs > - > > ambari-web/app/controllers/main/service/reassign/step4_controller.js > 840bda3 > ambari-web/test/controllers/main/service/reassign/step4_controller_test.js > 1e137ee > > Diff: https://reviews.apache.org/r/46800/diff/ > > > Testing > --- > > yes. > > 10505 tests complete (13 seconds) > 121 tests pending > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 29.607 s > [INFO] Finished at: 2016-04-28T14:53:42-07:00 > [INFO] Final Memory: 11M/213M > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On April 28, 2016, 10:54 p.m., Alexander Denissov wrote: > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, > > line 110 > > <https://reviews.apache.org/r/44210/diff/3-5/?file=1363971#file1363971line110> > > > > if we make isLocalHost utility function available in another utility, > > then there will be no need to pass stackAdvisor as parameter to all the > > functions in a ServiceAdvisor, as it promotes "spagetti-code", since > > StackAdvisor calls into ServiceAdvisor and having the call coming in the > > opposite directions is not desired. > > Tim Thorpe wrote: > I have been asked to keep this patch as minimal as possible to avoid > destabilizing the code. I could have refactored many of the functions into a > common file which both the stacks/stack_advisor.py and > stacks/service_advisor.py could use but this would result in many more > possible issues. To play it safe I passed in the stackAdvisor. In a > previous iteration, when I had refactored most of the stack advisor even for > the stack services like YARN, HDFS and HBASE, I found that sometimes I needed > information in say the YARN service_advisor.py that was only available in the > HDFS service_advisor.py. In that case, I would create an HDFS service > advisor and call the required functions. Now with all of those functions > being in the stackAdvisor, you can't do those sorts of things. This means > that for some service, you might need a reference to the stackAdvisor. > Obviously it needs to be used sparingly. > > In the future I hope to come back to this code and give a better > separation between stack and service advisors. Understood. I'm just afraid that the further rectoring might become problematic as it will require changes to the interface of the base service_advisor to remove passing in the stack advisor. Then we will have to refactor HAWq/PXF once again to consume the change. If the parameter is not there in the first place, service developers will take small steps to refactor pieces of logic they need from StackAdvisor into utils. However, we are OK with your approach as well. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131002 --- On April 28, 2016, 4:36 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > ------- > > (Updated April 28, 2016, 4:36 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-serve
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131020 --- Do we have a branch cut with these changes in so that we can test HAWQ and PXF with this new logic ? After testing the branch can be merged to trunk and we will avoid any major surprises. - Alexander Denissov On April 28, 2016, 4:36 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 28, 2016, 4:36 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/properties.json eac0dbd > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > d0ce196 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 2080c52 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131009 --- ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java (line 56) <https://reviews.apache.org/r/44210/#comment194940> should we not have a convention at all and rely on property specified in metainfo.xml ? The service advisors can be called HAWQ200ServiceAdvisor for the service in common services and HDP23HAWQ200ServiceAdvisor for the version specific to HDP2.3 stack, if any. This is the argument about inheritance between advisors of different versions of service and stack. ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java (line 65) <https://reviews.apache.org/r/44210/#comment194944> In case the service comes from the common services (like HAWQ) which can be versioned independently from the stack, I think we should include the service version: HDP23HAWQ200ServiceAdvisor when HDP2.3 uses HAWQ 2.0.0 HDP23HAWQ210ServiceAdvisor when HDP2.3 uses HAWQ 2.1.0 This example assumes that HAWQ 2.1.0 can be release before the next version of Ambari or stack can be released. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 33) <https://reviews.apache.org/r/44210/#comment194933> Should the name include the version, so the future versions can extend this: HAWQ200ServiceAdvisor ? Also applies to the file name, should we have HAWQ200_service_advisor.py ? ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 46) <https://reviews.apache.org/r/44210/#comment194952> we have a case (not coded yet) that might require more flexible lgoic here for which we will need more metadata. When we do thin, we might extend the interface to take a list of current component layout. - Alexander Denissov On April 28, 2016, 4:36 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 28, 2016, 4:36 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/properties.json eac0dbd > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > d0ce196 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/ma
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131002 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 110) <https://reviews.apache.org/r/44210/#comment194922> if we make isLocalHost utility function available in another utility, then there will be no need to pass stackAdvisor as parameter to all the functions in a ServiceAdvisor, as it promotes "spagetti-code", since StackAdvisor calls into ServiceAdvisor and having the call coming in the opposite directions is not desired. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 115) <https://reviews.apache.org/r/44210/#comment194921> since ServiceAdvisor should be called only when the service is installed or is chosen to be installed, do we really need the extra check here ? - Alexander Denissov On April 28, 2016, 4:36 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 28, 2016, 4:36 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/properties.json eac0dbd > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > d0ce196 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 2080c52 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 46742: Use additional flags for HAWQ Check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46742/#review130816 --- Please add unit tests. - Alexander Denissov On April 27, 2016, 5:29 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46742/ > --- > > (Updated April 27, 2016, 5:29 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16106 > https://issues.apache.org/jira/browse/AMBARI-16106 > > > Repository: ambari > > > Description > --- > > Make changes to HAWQ Check implementation based on HAWQ-668 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > 15ceaa2 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py > 76a5885 > > Diff: https://reviews.apache.org/r/46742/diff/ > > > Testing > --- > > Manually tested > > Ran unit tests: > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok > > -- > Ran 35 tests in 0.126s > > OK > > > Thanks, > > Matt > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote: > > So, what will be the rules for SA logic inheritance ? When MYSERVICE/2.0.0 ships and needs to reuse the stack_advisor logic from MYSERVICE/1.0.0 -- will we: - rely on stack inheritance ? - rely on SA inheritance within MYSERVICE ? - combination of these ? - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review130141 --- On April 22, 2016, 6:27 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 22, 2016, 6:27 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > d574d60 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > db95fec > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/properties.json eac0dbd > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 0130483 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > 3a65541 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25 > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 6c9fd46 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 46509: Remove performing service check during "Remove Standby Wizard"
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46509/#review130162 --- Ship it! Ship It! - Alexander Denissov On April 21, 2016, 11:01 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46509/ > --- > > (Updated April 21, 2016, 11:01 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16024 > https://issues.apache.org/jira/browse/AMBARI-16024 > > > Repository: ambari > > > Description > --- > > Users will need to remove the HAWQ Standby Master using "Remove Standby > Wizard" after enabling HDFS HA. > So the service check may fail during the wizard as the HAWQ catalog might be > still pointing to the old filespace. > > > Diffs > - > > > ambari-web/app/controllers/main/admin/highAvailability/hawq/removeStandby/step3_controller.js > 5bcf34f > > Diff: https://reviews.apache.org/r/46509/diff/ > > > Testing > --- > > yes. > 25660 tests complete (29 seconds) > 154 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 46.511 s > [INFO] Finished at: 2016-04-21T16:03:11-07:00 > [INFO] Final Memory: 14M/331M > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 46359: HAWQ Immediate Stop cluster fails with key error
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46359/#review129485 --- Ship it! Ship It! - Alexander Denissov On April 19, 2016, 1:20 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46359/ > --- > > (Updated April 19, 2016, 1:20 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-15960 > https://issues.apache.org/jira/browse/AMBARI-15960 > > > Repository: ambari > > > Description > --- > > Error log from running the wizard: > > > stderr: > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py", > line 98, in > HawqMaster().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 219, in execute > method(env) > File > "/var/lib/ambari-agent/cache/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py", > line 69, in immediate_stop_hawq_service > common.stop_component(hawq_constants.CLUSTER, hawq_constants.IMMEDIATE) > File > "/var/lib/ambari-agent/cache/common-services/HAWQ/2.0.0/package/scripts/common.py", > line 309, in stop_component > port_property_name = > hawq_constants.COMPONENT_ATTRIBUTES_MAP[component_name]['port_property'] > KeyError: 'cluster' > stdout: > None > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py > 4ce0c94 > > Diff: https://reviews.apache.org/r/46359/diff/ > > > Testing > --- > > Manually Tested. Unit Tests passed. > > > Thanks, > > Matt > >
Re: Review Request 46294: HAWQ activate standby wizard fails after port number change but before restart.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46294/#review129412 --- Ship it! Ship It! - Alexander Denissov On April 16, 2016, 12:56 a.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46294/ > --- > > (Updated April 16, 2016, 12:56 a.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15926 > https://issues.apache.org/jira/browse/AMBARI-15926 > > > Repository: ambari > > > Description > --- > > Activating standby could start a master with an old port, and the following > operations in the Activate Standby wizard could be mislead. > The fix is to stop the newly become master so that the wizard will only have > to deal with the current value. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > e2114d8 > > Diff: https://reviews.apache.org/r/46294/diff/ > > > Testing > --- > > Unit test passed. > Manually tested. > > > Thanks, > > jun aoki > >
Re: Review Request 46186: Changing HAWQ Ports through Ambari prevents HAWQ service from restarting
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46186/#review129024 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstatus.py (line 31) <https://reviews.apache.org/r/46186/#comment192489> variable is not used anywhere else other than the line below, can probably inline it for brevity ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py (line 26) <https://reviews.apache.org/r/46186/#comment192490> do we need unit test for this function ? - Alexander Denissov On April 14, 2016, 9:20 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46186/ > --- > > (Updated April 14, 2016, 9:20 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15852 > https://issues.apache.org/jira/browse/AMBARI-15852 > > > Repository: ambari > > > Description > --- > > HAWQ restart after port change will fail because Ambari looks for a currently > running process with a new port number. > This change is to make Ambari to look for an older port value. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 413cf1a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py > 3f6c371 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > f2ec914 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py > 636dcad > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > 67d1cb0 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstatus.py > 92ac0af > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py > 0baa4c5 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py c18941c > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 17d7473 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 44565b3 > > Diff: https://reviews.apache.org/r/46186/diff/ > > > Testing > --- > > Unit test has passed. > Manual Test in progress. > > > Thanks, > > jun aoki > >
Review Request 46062: PXF service check must occur only after PXF is started
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46062/ --- Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Jayush Luniya, Lav Jain, Matt, Sumit Mohanty, and Srimanth Gunturi. Bugs: AMBARI-15817 https://issues.apache.org/jira/browse/AMBARI-15817 Repository: ambari Description --- PXF service check must occur only after PXF is started Diffs - ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json b179d1c Diff: https://reviews.apache.org/r/46062/diff/ Testing --- Tested manually, ran tests: -- Ran 246 tests in 7.222s OK -- Total run:925 Total errors:0 Total failures:0 Thanks, Alexander Denissov
Review Request 45826: Descriptions for HAWQ sysctl params mention incorrect default values
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45826/ --- Review request for Ambari, bhuvnesh chaudhary, Goutam Tadi, Lav Jain, and Matt. Bugs: AMBARI-15738 https://issues.apache.org/jira/browse/AMBARI-15738 Repository: ambari Description --- Descriptions for HAWQ sysctl params mention incorrect default values kernel.shmmax and net.ipv4.ip_local_port_range have incorrect default values in their descriptions Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml 365fa86 Diff: https://reviews.apache.org/r/45826/diff/ Testing --- Manual, tests ran OK -- Ran 246 tests in 7.005s OK -- Total run:921 Total errors:0 Total failures:0 Thanks, Alexander Denissov
Review Request 45303: Update HAWQ default_hash_table_bucket_number property logic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45303/ --- Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, Jayush Luniya, Lav Jain, Matt, Oleksandr Diachenko, and Sumit Mohanty. Bugs: AMBARI-15566 https://issues.apache.org/jira/browse/AMBARI-15566 Repository: ambari Description --- Update HAWQ default_hash_table_bucket_number property logic Diffs - ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 4635199 ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 682e1c2 Diff: https://reviews.apache.org/r/45303/diff/ Testing --- manual -- Ran 246 tests in 7.045s OK -- Total run:911 Total errors:0 Total failures:0 Thanks, Alexander Denissov
Re: Review Request 44868: Changed HAWQ directory descriptions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44868/ --- (Updated March 16, 2016, 10:42 p.m.) Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Jayush Luniya, Lav Jain, Oleksandr Diachenko, and Sumit Mohanty. Changes --- updated description sentences Bugs: AMBARI-15433 https://issues.apache.org/jira/browse/AMBARI-15433 Repository: ambari Description --- Changed HAWQ directory descriptions Diffs (updated) - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml d3aa0ed Diff: https://reviews.apache.org/r/44868/diff/ Testing --- manual. Thanks, Alexander Denissov
Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44987/#review124115 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py (line 60) <https://reviews.apache.org/r/44987/#comment186491> maybe pass the mode into create_dir_as_hawq_user (default None) so it can be used with Directory(mode=xyz) resource and not via chmod ? Then there will be no need for this function at all. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py (line 66) <https://reviews.apache.org/r/44987/#comment186492> I wonder if Ambari already supports creating a list of directories from a string with comma-separated values via Directory resource ? ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py (line 45) <https://reviews.apache.org/r/44987/#comment186493> plural, so perhaps "..they should be.." instead of "..it should be.." - Alexander Denissov On March 17, 2016, 10:37 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44987/ > --- > > (Updated March 17, 2016, 10:37 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15475 > https://issues.apache.org/jira/browse/AMBARI-15475 > > > Repository: ambari > > > Description > --- > > Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many > [/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment] > and the change is not propagated to segments and the folders are not created. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > d62332d > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py > 0a597b6 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 9482b35 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py > 1bdc5aa > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 > > Diff: https://reviews.apache.org/r/44987/diff/ > > > Testing > --- > > manually tested. > > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 > 15:33:42,969 - Standby host c6401.ambari.apache.org initialized > ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 > 15:33:43,025 - Active master c6401.ambari.apache.org initialized > ok > test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 > 15:33:43,049 - Master c6401.ambari.apache.org started > ok > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > > > Thanks, > > jun aoki > >
Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property
> On March 18, 2016, 5:08 p.m., jun aoki wrote: > > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py, > > line 699 > > <https://reviews.apache.org/r/45012/diff/1/?file=1304803#file1304803line699> > > > > do you want to inline this since propertyvalue is used only once. I prefer to leave it separate for clarity > On March 18, 2016, 5:08 p.m., jun aoki wrote: > > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py, > > line 984 > > <https://reviews.apache.org/r/45012/diff/1/?file=1304803#file1304803line984> > > > > Do you want to set (constant) variable for a magic number 4? > > ``` > > REPLACE_DATANODE_ON_FAILURE_THREASHOLD = 4 # my poor naming skill... > > if numSegments > REPLACE_DATANODE_ON_FAILURE_THREASHOLD and value ... : > > ... > > elif numSegments <= REPLACE_DATANODE_ON_FAILURE_THREASHOLD and value > > ... : > > ... > > > > ``` > > > > and you can use it on line 694 added a constant, but not for the whole file, but rather within the method. I feel that exposing the constant for the whole file is too intrusive. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45012/#review124221 --- On March 18, 2016, 7:47 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45012/ > --- > > (Updated March 18, 2016, 7:47 p.m.) > > > Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Jayush Luniya, Lav Jain, and Sumit Mohanty. > > > Bugs: AMBARI-15499 > https://issues.apache.org/jira/browse/AMBARI-15499 > > > Repository: ambari > > > Description > --- > > Recommendation and validation logic for HAWQ hdfs-client.xml > output.replace-datanode-on-failure property > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 93ad3fe > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml > c2a756a > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > c0bc4e2 > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 4f0b4b7 > > Diff: https://reviews.apache.org/r/45012/diff/ > > > Testing > --- > > manual > > -- > Ran 253 tests in 7.111s > > OK > -- > Total run:927 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Alexander Denissov > >
Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45012/ --- (Updated March 18, 2016, 7:34 p.m.) Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Jayush Luniya, Lav Jain, and Sumit Mohanty. Changes --- changed to lower-case booleans, updated some descriptions Bugs: AMBARI-15499 https://issues.apache.org/jira/browse/AMBARI-15499 Repository: ambari Description --- Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property Diffs (updated) - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml 93ad3fe ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml c2a756a ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py c0bc4e2 ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 4f0b4b7 Diff: https://reviews.apache.org/r/45012/diff/ Testing --- manual -- Ran 253 tests in 7.111s OK -- Total run:927 Total errors:0 Total failures:0 OK Thanks, Alexander Denissov
