Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/#review136151
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 179)


Minor! Line 179 can be moved next to line 212.


- Lav Jain


On June 3, 2016, 10:49 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48235/
> ---
> 
> (Updated June 3, 2016, 10:49 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-17023
> https://issues.apache.org/jira/browse/AMBARI-17023
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show only relevant properties based on the status of HAWQ Resource Manager.
> When Yarn mode is enable, only two parameters should be invisible.
> hawq_rm_memory_limit_perseg
> hawq_rm_nvcore_limit_perseg
> When standalone mode is enabled, four parameters should be invisible. what 
> you list are correct.
> hawq_rm_yarn_app_name
> hawq_rm_yarn_queue_name
> hawq_rm_yarn_scheduler_address
> hawq_rm_yarn_address
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
> 
> Diff: https://reviews.apache.org/r/48235/diff/
> 
> 
> Testing
> ---
> 
> yes.
> Now testing 
> /Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
> ~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
>  ~/github/ambari-toolbox/vagrant/ambari
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-06-03 15:50:22,301 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-06-03 15:50:22,307 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 10: HDFS-HA, YARN Resource 

Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/#review136150
---




ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 


hawq_rm_nvcore_limit_perseg (same for hawq_rm_memory_limit_perseg) should 
be set only when rm_type is none. Any particular reason for removing this test?


- Lav Jain


On June 3, 2016, 10:49 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48235/
> ---
> 
> (Updated June 3, 2016, 10:49 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-17023
> https://issues.apache.org/jira/browse/AMBARI-17023
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show only relevant properties based on the status of HAWQ Resource Manager.
> When Yarn mode is enable, only two parameters should be invisible.
> hawq_rm_memory_limit_perseg
> hawq_rm_nvcore_limit_perseg
> When standalone mode is enabled, four parameters should be invisible. what 
> you list are correct.
> hawq_rm_yarn_app_name
> hawq_rm_yarn_queue_name
> hawq_rm_yarn_scheduler_address
> hawq_rm_yarn_address
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
> 
> Diff: https://reviews.apache.org/r/48235/diff/
> 
> 
> Testing
> ---
> 
> yes.
> Now testing 
> /Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
> ~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
>  ~/github/ambari-toolbox/vagrant/ambari
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-06-03 15:50:22,301 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-06-03 15:50:22,307 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case10 

Re: Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48234/#review136145
---


Ship it!




Ship It!

- Nate Cole


On June 3, 2016, 6:47 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48234/
> ---
> 
> (Updated June 3, 2016, 6:47 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17037
> https://issues.apache.org/jira/browse/AMBARI-17037
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fresh installation and EU/RU of Falcon is failing.
> In HDP 2.4, Falcon used the data mirroring folder.
> In HDP 2.5, data mirroring is no longer used and instead uses the extensions 
> folder that must be uploaded to HDFS.
> 
> This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions 
> folder to HDFS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  4c13aaa 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  5e25325 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
>  7c2cdf4 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  8c2ad8e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 
> 
> Diff: https://reviews.apache.org/r/48234/diff/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1052
> Total errors:0
> Total failures:0
> OK
> 
> Express Upgrade is currently broken, so will commit this patch and retest 
> once it is stable again in Hadoop Core.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 48242: Takes long time to start or fail to start service after enabling SSL due to "dfs.https.enable"

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48242/
---

Review request for Ambari, Alejandro Fernandez and Andrew Onischuk.


Bugs: AMBARI-17039
https://issues.apache.org/jira/browse/AMBARI-17039


Repository: ambari


Description
---

Check both dfs.http.policy and deprecated dfs.https.enable


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdfs_utils.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py
 919ccb5 
  
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 de46c1c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 b634ce9 

Diff: https://reviews.apache.org/r/48242/diff/


Testing
---

mvn clean test


Thanks,

Richard Zang

Re: Review Request 48241: Enable kerberos wizard UI showing incorrect total of required fields

[Yusaku Sako]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48241/#review136139
---


Ship it!




Ship It!

- Yusaku Sako


On June 3, 2016, 11:43 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48241/
> ---
> 
> (Updated June 3, 2016, 11:43 p.m.)
> 
> 
> Review request for Ambari, Zhe (Joe) Wang, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17038
> https://issues.apache.org/jira/browse/AMBARI-17038
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Incorrect total of required fields on enable kerberos wizard
> Total fields 7 but total on kerberos tab showing 5.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/configs/objects/service_config.js 088d51e 
>   ambari-web/test/models/configs/objects/service_config_test.js 70cf37b 
> 
> Diff: https://reviews.apache.org/r/48241/diff/
> 
> 
> Testing
> ---
> 
> ambari-web unit tests passes with the patch:
> 28646 tests complete (29 seconds)
> 154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Review Request 48241: Enable kerberos wizard UI showing incorrect total of required fields

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48241/
---

Review request for Ambari, Zhe (Joe) Wang, Xi Wang, and Yusaku Sako.


Bugs: AMBARI-17038
https://issues.apache.org/jira/browse/AMBARI-17038


Repository: ambari


Description
---

Incorrect total of required fields on enable kerberos wizard
Total fields 7 but total on kerberos tab showing 5.


Diffs
-

  ambari-web/app/models/configs/objects/service_config.js 088d51e 
  ambari-web/test/models/configs/objects/service_config_test.js 70cf37b 

Diff: https://reviews.apache.org/r/48241/diff/


Testing
---

ambari-web unit tests passes with the patch:
28646 tests complete (29 seconds)
154 tests pending


Thanks,

Jaimin Jetly

Re: Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48234/#review136136
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
 (line 92)


How could upgrade_type be None in "pre_upgrade"restart" ?



ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 (line 73)


Maybe use os.path.join here for these?


- Jonathan Hurley


On June 3, 2016, 6:47 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48234/
> ---
> 
> (Updated June 3, 2016, 6:47 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17037
> https://issues.apache.org/jira/browse/AMBARI-17037
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fresh installation and EU/RU of Falcon is failing.
> In HDP 2.4, Falcon used the data mirroring folder.
> In HDP 2.5, data mirroring is no longer used and instead uses the extensions 
> folder that must be uploaded to HDFS.
> 
> This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions 
> folder to HDFS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  4c13aaa 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  5e25325 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
>  7c2cdf4 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  8c2ad8e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 
> 
> Diff: https://reviews.apache.org/r/48234/diff/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1052
> Total errors:0
> Total failures:0
> OK
> 
> Express Upgrade is currently broken, so will commit this patch and retest 
> once it is stable again in Hadoop Core.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 48205: Cluster operator and ServiceAdministrator not allowed to create config group

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48205/#review136135
---


Ship it!




Ship It!

- Jonathan Hurley


On June 3, 2016, 9:56 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48205/
> ---
> 
> (Updated June 3, 2016, 9:56 a.m.)
> 
> 
> Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Myroslav 
> Papirkovskyy, and Nate Cole.
> 
> 
> Bugs: AMBARI-17029
> https://issues.apache.org/jira/browse/AMBARI-17029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Cluster operator and ServiceAdministrator are not allowed to create a config 
> group even though it should be.
> It is visible in the UI but creating it gives a 403 error in the backend and 
> the UI goes for a toss.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
>  6fe29db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
>  2b9f304 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  d56ed44 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
>  2913cf5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/TestAuthenticationFactory.java
>  4301bf8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java
>  96b2cfb 
> 
> Diff: https://reviews.apache.org/r/48205/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> #Local test results: 
> Unrelated unit test failures:
> ```
> Results :
> 
> Tests in error:
>   StackManagerTest.testMetricsLoaded:669 » Ambari File 
> /Users/rlevas/git/ambari/...
>   StackManagerTest.testServicesWithLogsearchRoleCommandOrder:820 » Ambari 
> File /...
>   StackManagerTest.testServicesWithRangerPluginRoleCommandOrder:713 » Ambari 
> Fil...
>   ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /Us...
> 
> Tests run: 4458, Failures: 0, Errors: 4, Skipped: 34
> ```
> 
> #Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136134
---


Ship it!




Ship It!

- Robert Levas


On June 3, 2016, 5:30 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
> 
> (Updated June 3, 2016, 5:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When retrieving JMX properties on large clusters, if there are JMX endpoints 
> which are unreachable, the URI protocol value is never cached. As a result, 
> multiple attempts are made to get the JMX protocol. Normally, this would be 
> fine, except that retrieving the configuration is using the 
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
> 
> {code}
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
>   at 
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
>   at 
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
>   at 
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> 

Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/#review136131
---


Ship it!




Ship It!

- Matt


On June 3, 2016, 3:49 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48235/
> ---
> 
> (Updated June 3, 2016, 3:49 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-17023
> https://issues.apache.org/jira/browse/AMBARI-17023
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show only relevant properties based on the status of HAWQ Resource Manager.
> When Yarn mode is enable, only two parameters should be invisible.
> hawq_rm_memory_limit_perseg
> hawq_rm_nvcore_limit_perseg
> When standalone mode is enabled, four parameters should be invisible. what 
> you list are correct.
> hawq_rm_yarn_app_name
> hawq_rm_yarn_queue_name
> hawq_rm_yarn_scheduler_address
> hawq_rm_yarn_address
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
> 
> Diff: https://reviews.apache.org/r/48235/diff/
> 
> 
> Testing
> ---
> 
> yes.
> Now testing 
> /Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
> ~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
>  ~/github/ambari-toolbox/vagrant/ambari
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-06-03 15:50:22,301 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-06-03 15:50:22,307 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
> Kerberized ... 2016-06-03 15:50:22,321 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check 

Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[Alexander Denissov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/#review136128
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 182)


better to move these lines higher to after line 149 where the similar logic 
resides ?



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 190)


are we repeating default values already defined in .xml file ? Would be 
better not to do that, if possible.


- Alexander Denissov


On June 3, 2016, 10:49 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48235/
> ---
> 
> (Updated June 3, 2016, 10:49 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-17023
> https://issues.apache.org/jira/browse/AMBARI-17023
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show only relevant properties based on the status of HAWQ Resource Manager.
> When Yarn mode is enable, only two parameters should be invisible.
> hawq_rm_memory_limit_perseg
> hawq_rm_nvcore_limit_perseg
> When standalone mode is enabled, four parameters should be invisible. what 
> you list are correct.
> hawq_rm_yarn_app_name
> hawq_rm_yarn_queue_name
> hawq_rm_yarn_scheduler_address
> hawq_rm_yarn_address
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
> 
> Diff: https://reviews.apache.org/r/48235/diff/
> 
> 
> Testing
> ---
> 
> yes.
> Now testing 
> /Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
> ~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
>  ~/github/ambari-toolbox/vagrant/ambari
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-06-03 15:50:22,301 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> 

Re: Review Request 47673: Wrong memory conf in spark-env.xml

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47673/#review136129
---




ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml
 (line 92)


For existing clusters with HDP 2.x, should they change this value when they 
upgrade Ambari and/or the stack to HDP 2.(x+1)?


- Alejandro Fernandez


On June 3, 2016, 11:01 p.m., Weiqing Yang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47673/
> ---
> 
> (Updated June 3, 2016, 11:01 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16715
> https://issues.apache.org/jira/browse/AMBARI-16715
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In spark-env.xml, the memory value should not have space and a extra "b" 
> character. "SPARK_DRIVER_MEMORY="512 Mb" #Memory for Master (e.g. 1000M, 2G) 
> (Default: 512 Mb)", by default this line is commented but in case user 
> uncommented it to change the memory settings then it will show a below error, 
> so it's better to change it in the default template.
> Error:
> ./bin/spark-shell
> Error: Could not find or load main class Mb
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml
>  34c5462 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml
>  c5fcc85 
> 
> Diff: https://reviews.apache.org/r/47673/diff/
> 
> 
> Testing
> ---
> 
> N/A
> 
> 
> Thanks,
> 
> Weiqing Yang
> 
>

Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[bhuvnesh chaudhary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/
---

Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.


Bugs: AMBARI-17023
https://issues.apache.org/jira/browse/AMBARI-17023


Repository: ambari


Description
---

Show only relevant properties based on the status of HAWQ Resource Manager.
When Yarn mode is enable, only two parameters should be invisible.
hawq_rm_memory_limit_perseg
hawq_rm_nvcore_limit_perseg
When standalone mode is enabled, four parameters should be invisible. what you 
list are correct.
hawq_rm_yarn_app_name
hawq_rm_yarn_queue_name
hawq_rm_yarn_scheduler_address
hawq_rm_yarn_address


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 3ac4e89 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
28eb82f 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 5750938 

Diff: https://reviews.apache.org/r/48235/diff/


Testing
---

yes.
Now testing 
/Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
 ~/github/ambari-toolbox/vagrant/ambari
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
HAWQ Standby Master ...
ok
test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Resync HAWQ Standby ... 2016-06-03 15:50:22,301 - HAWQ 
Standby Master Re-Sync started in fast mode...
ok
test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-06-03 15:50:22,307 - Executing HAWQ Check ...
ok
test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-06-03 15:50:22,321 - Executing HAWQ Check ...
ok
test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
Kerberized ... 2016-06-03 15:50:22,333 - Executing HAWQ Check ...
ok
test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
Kerberized ... 2016-06-03 15:50:22,346 - Executing HAWQ Check ...
ok
test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 2: Non HDFS-HA, Standalone Resource Management, 
Kerberized ... 2016-06-03 15:50:22,359 - Executing HAWQ Check ...
ok
test_run_hawq_check_case3 (test_hawqmaster.TestHawqMaster)

Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48234/
---

Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe.


Bugs: AMBARI-17037
https://issues.apache.org/jira/browse/AMBARI-17037


Repository: ambari


Description
---

Fresh installation and EU/RU of Falcon is failing.
In HDP 2.4, Falcon used the data mirroring folder.
In HDP 2.5, data mirroring is no longer used and instead uses the extensions 
folder that must be uploaded to HDFS.

This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions 
folder to HDFS


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 34d1a1a 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 4c13aaa 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 5e25325 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
 7c2cdf4 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 8c2ad8e 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 8b669e8 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 

Diff: https://reviews.apache.org/r/48234/diff/


Testing
---

Python unit tests passed.

--
Total run:1052
Total errors:0
Total failures:0
OK

Express Upgrade is currently broken, so will commit this patch and retest once 
it is stable again in Hadoop Core.


Thanks,

Alejandro Fernandez

Re: Review Request 48205: Cluster operator and ServiceAdministrator not allowed to create config group

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48205/#review136118
---


Ship it!




Ship It!

- Nate Cole


On June 3, 2016, 9:56 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48205/
> ---
> 
> (Updated June 3, 2016, 9:56 a.m.)
> 
> 
> Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Myroslav 
> Papirkovskyy, and Nate Cole.
> 
> 
> Bugs: AMBARI-17029
> https://issues.apache.org/jira/browse/AMBARI-17029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Cluster operator and ServiceAdministrator are not allowed to create a config 
> group even though it should be.
> It is visible in the UI but creating it gives a 403 error in the backend and 
> the UI goes for a toss.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
>  6fe29db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
>  2b9f304 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  d56ed44 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
>  2913cf5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/TestAuthenticationFactory.java
>  4301bf8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java
>  96b2cfb 
> 
> Diff: https://reviews.apache.org/r/48205/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> #Local test results: 
> Unrelated unit test failures:
> ```
> Results :
> 
> Tests in error:
>   StackManagerTest.testMetricsLoaded:669 » Ambari File 
> /Users/rlevas/git/ambari/...
>   StackManagerTest.testServicesWithLogsearchRoleCommandOrder:820 » Ambari 
> File /...
>   StackManagerTest.testServicesWithRangerPluginRoleCommandOrder:713 » Ambari 
> Fil...
>   ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /Us...
> 
> Tests run: 4458, Failures: 0, Errors: 4, Skipped: 34
> ```
> 
> #Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/#review136105
---



This is a fairly big patch and I'm surprised more people haven't reviewed it.
Please send an email to the community, including Nate Cole and Jonathan Hurley


ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
 (line 27)


Add javadoc for all new classes



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 (line 3798)


Add javadoc to new methods
Someone new reading the code should know what an extension is.



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 (line 4986)


Can store the string and use it for the LOG and to throw the exception



ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
 (line 124)


Should the stack name also be part of the hash?



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
 (line 30)


Remove unused imports



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
 (line 108)


Remove unused code



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
 (line 21)


Use named imports



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
 (line 182)


Remove dead code



ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
(line 521)


Remove dead code



ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql (line 32)


Let's be consistent with the name, UQ_*


- Alejandro Fernandez


On June 3, 2016, 4:54 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47656/
> ---
> 
> (Updated June 3, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth 
> Gunturi, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12885
> https://issues.apache.org/jira/browse/AMBARI-12885
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The purpose of this proposal is to facilitate adding custom services to an 
> existing stack. Ideally this would support adding and upgrading custom 
> services separately from the core services defined in the stack. In 
> particular we are looking at custom services that need to support several 
> different stacks (different distributions of Ambari). The release cycle of 
> the custom services may be different from that of the core stack; that is, a 
> custom service may be upgraded at a different rate than the core distribution 
> itself and may be upgraded multiple times within the lifespan of a single 
> release of the core distribution.
> 
> One possible approach to handling this would be dynamically extending a stack 
> (after install time). It would be best to extend the stack in packages where 
> a stack extension package can have one or more custom services.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> fc1b72a 
>   ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
>   ambari-server/conf/unix/ambari.properties a88a025 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  cf7c391 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  f0928cf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
>  PRE-CREATION 
>   
> 

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136112
---


Ship it!




Ship It!

- Nate Cole


On June 3, 2016, 5:30 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
> 
> (Updated June 3, 2016, 5:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When retrieving JMX properties on large clusters, if there are JMX endpoints 
> which are unreachable, the URI protocol value is never cached. As a result, 
> multiple attempts are made to get the JMX protocol. Normally, this would be 
> fine, except that retrieving the configuration is using the 
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
> 
> {code}
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
>   at 
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
>   at 
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
>   at 
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> 

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Jonathan Hurley]

> On June 3, 2016, 5:09 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java,
> >  line 1201
> > 
> >
> > one day we'll support multi-cluster, so maybe this should combine 
> > cluster name + component

It didn't work before ... working 99% of the time is a lot better :)

Nice catch; I'll fix it.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136108
---


On June 3, 2016, 5:30 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
> 
> (Updated June 3, 2016, 5:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When retrieving JMX properties on large clusters, if there are JMX endpoints 
> which are unreachable, the URI protocol value is never cached. As a result, 
> multiple attempts are made to get the JMX protocol. Normally, this would be 
> fine, except that retrieving the configuration is using the 
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
> 
> {code}
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
>   at 
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
>   at 
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
>   at 
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
>   at 
> 

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/
---

(Updated June 3, 2016, 5:30 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Changes
---

Switched to using a Guava table as a way to bind two String keys to a single 
mapped value.


Bugs: AMBARI-17036
https://issues.apache.org/jira/browse/AMBARI-17036


Repository: ambari


Description
---

When retrieving JMX properties on large clusters, if there are JMX endpoints 
which are unreachable, the URI protocol value is never cached. As a result, 
multiple attempts are made to get the JMX protocol. Normally, this would be 
fine, except that retrieving the configuration is using the 
{{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.

{code}
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
at 
com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at 
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at 
org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
at 
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at 
com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
at 
org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
at 
org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
at 
org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
at 
org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
at 
org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
at 
org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
at 
org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 591441b 
  

Re: Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/
---

(Updated June 3, 2016, 9:26 p.m.)


Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
Luniya, Matt, and Tim Thorpe.


Changes
---

Rebased with branch-2.4


Bugs: AMBARI-16663
https://issues.apache.org/jira/browse/AMBARI-16663


Repository: ambari


Description
---

The original intent was to move the common functions that are used by both 
stack and service advisors to a separate utils file to avoid passing 
stack_advisor class into service advisor apis.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
28eb82f 
  ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
9b34171 
  ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
  ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 5750938 
  ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py c3a63cc 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 424a386 

Diff: https://reviews.apache.org/r/48229/diff/


Testing
---

Lavs-MacBook-Pro:common ljain$ python -m discover -v
test_createComponentLayoutRecommendations_hawq_1_Host 
(test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
for service HAWQ was loaded
[u'c6401.ambari.apache.org']
[u'c6401.ambari.apache.org']
ok
test_createComponentLayoutRecommendations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
implementation for service HAWQ was loaded
[u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
u'c6403.ambari.apache.org']
[u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
u'c6403.ambari.apache.org']
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT does not get recommended during Add Service Wizard, when 
HAWQ has already been installed ... ServiceAdvisor implementation for service 
HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Cluster Install Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components ... ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF does not get recommended during Add Service Wizard, when PXF has 
already been installed ... ServiceAdvisor implementation for service PXF was 
loaded
ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Add Service Wizard, when PXF is 
selected for installation ... ServiceAdvisor implementation for service PXF was 
loaded
ok
test_createComponentLayoutRecommendations_pxf_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Cluster Install Wizard, when 
PXF is selected for installation ... ServiceAdvisor implementation for service 
PXF was loaded
ok
test_getComponentLayoutValidations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test layout validations for HAWQ components on a 3-node cluster ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
(test_stack_advisor.TestHDP23StackAdvisor)
Test validation warning for HAWQ segment not colocated with DATANODE ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_nohawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components on a 3-node cluster ... ok
test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test NO warning is generated when PXF is co-located with NAMENODE and DATANODE 
... ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_pxf_not_co_located_with_dn 

Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/
---

Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
Luniya, Matt, and Tim Thorpe.


Bugs: AMBARI-16663
https://issues.apache.org/jira/browse/AMBARI-16663


Repository: ambari


Description
---

The original intent was to move the common functions that are used by both 
stack and service advisors to a separate utils file to avoid passing 
stack_advisor class into service advisor apis.


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
a634062 
  ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
9b34171 
  ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
  ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 7b64d04 
  ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py c3a63cc 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py b931dc6 

Diff: https://reviews.apache.org/r/48229/diff/


Testing
---

Lavs-MacBook-Pro:common ljain$ python -m discover -v
test_createComponentLayoutRecommendations_hawq_1_Host 
(test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
for service HAWQ was loaded
[u'c6401.ambari.apache.org']
[u'c6401.ambari.apache.org']
ok
test_createComponentLayoutRecommendations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
implementation for service HAWQ was loaded
[u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
u'c6403.ambari.apache.org']
[u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
u'c6403.ambari.apache.org']
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT does not get recommended during Add Service Wizard, when 
HAWQ has already been installed ... ServiceAdvisor implementation for service 
HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Cluster Install Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components ... ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF does not get recommended during Add Service Wizard, when PXF has 
already been installed ... ServiceAdvisor implementation for service PXF was 
loaded
ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Add Service Wizard, when PXF is 
selected for installation ... ServiceAdvisor implementation for service PXF was 
loaded
ok
test_createComponentLayoutRecommendations_pxf_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Cluster Install Wizard, when 
PXF is selected for installation ... ServiceAdvisor implementation for service 
PXF was loaded
ok
test_getComponentLayoutValidations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test layout validations for HAWQ components on a 3-node cluster ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
(test_stack_advisor.TestHDP23StackAdvisor)
Test validation warning for HAWQ segment not colocated with DATANODE ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_nohawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components on a 3-node cluster ... ok
test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test NO warning is generated when PXF is co-located with NAMENODE and DATANODE 
... ServiceAdvisor implementation for service PXF was loaded
ok
test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test warning is generated when PXF is not co-located with NAMENODE or DATANODE 

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136108
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 (line 1175)


one day we'll support multi-cluster, so maybe this should combine cluster 
name + component


- Nate Cole


On June 3, 2016, 4:56 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
> 
> (Updated June 3, 2016, 4:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When retrieving JMX properties on large clusters, if there are JMX endpoints 
> which are unreachable, the URI protocol value is never cached. As a result, 
> multiple attempts are made to get the JMX protocol. Normally, this would be 
> fine, except that retrieving the configuration is using the 
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
> 
> {code}
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
>   at 
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
>   at 
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
>   at 
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
>   at 
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/
---

(Updated June 3, 2016, 4:56 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Changes
---

Tests updated


Bugs: AMBARI-17036
https://issues.apache.org/jira/browse/AMBARI-17036


Repository: ambari


Description
---

When retrieving JMX properties on large clusters, if there are JMX endpoints 
which are unreachable, the URI protocol value is never cached. As a result, 
multiple attempts are made to get the JMX protocol. Normally, this would be 
fine, except that retrieving the configuration is using the 
{{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.

{code}
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
at 
com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at 
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at 
org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
at 
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at 
com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
at 
org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
at 
org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
at 
org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
at 
org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
at 
org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
at 
org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
at 
org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 591441b 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
488603a 
  

Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136101
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 (lines 981 - 997)


I have no idea why in the world this was being done this way. There's no 
need to go through a ClusterResourceProvider for the tag of the currently 
desired version for a config. 

It was causing a massive amount of JSON and processing and DB hits. 

Instead, just go directly to the Cluster instance to get that data.


- Jonathan Hurley


On June 3, 2016, 4:06 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
> 
> (Updated June 3, 2016, 4:06 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When retrieving JMX properties on large clusters, if there are JMX endpoints 
> which are unreachable, the URI protocol value is never cached. As a result, 
> multiple attempts are made to get the JMX protocol. Normally, this would be 
> fine, except that retrieving the configuration is using the 
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
> 
> {code}
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
>   at 
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
>   at 
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>   at 
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
>   at 
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
>   at 
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
>   at 
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
>   at 
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
>   at 
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
>   at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
>   at 
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
>   at 
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
>   at 
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
>   at 
> 

Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-17036
https://issues.apache.org/jira/browse/AMBARI-17036


Repository: ambari


Description
---

When retrieving JMX properties on large clusters, if there are JMX endpoints 
which are unreachable, the URI protocol value is never cached. As a result, 
multiple attempts are made to get the JMX protocol. Normally, this would be 
fine, except that retrieving the configuration is using the 
{{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.

{code}
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
at 
com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at 
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at 
org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
at 
com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at 
com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at 
org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
at 
org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
at 
org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
at 
org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
at 
org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
at 
org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
at 
org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
at 
org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
at 
org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
at 
org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
at 
org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
at 
org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 591441b 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
488603a 

Diff: https://reviews.apache.org/r/48221/diff/


Testing
---

Pending...


Thanks,

Jonathan Hurley

Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

[Tim Thorpe]

> On June 3, 2016, 6:29 p.m., Alejandro Fernandez wrote:
> > A lof of files have changed, please give me 24 hours to go over this. Thank 
> > you

The patch really hasn't changed all that much in the past week or so.  I did 
add the Upgrade240 changes to add the 2 new DB tables when upgrading to 2.4 
during the last week.  Other than that it is just redoing the patch so that it 
can work over the latest trunk changes.

I will be doing a demo of this on June 7th @11am PST.  Contact Jayush if you 
want to be included in the meeting invite.  Thanks


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/#review136089
---


On June 3, 2016, 4:54 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47656/
> ---
> 
> (Updated June 3, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth 
> Gunturi, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12885
> https://issues.apache.org/jira/browse/AMBARI-12885
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The purpose of this proposal is to facilitate adding custom services to an 
> existing stack. Ideally this would support adding and upgrading custom 
> services separately from the core services defined in the stack. In 
> particular we are looking at custom services that need to support several 
> different stacks (different distributions of Ambari). The release cycle of 
> the custom services may be different from that of the core stack; that is, a 
> custom service may be upgraded at a different rate than the core distribution 
> itself and may be upgraded multiple times within the lifespan of a single 
> release of the core distribution.
> 
> One possible approach to handling this would be dynamically extending a stack 
> (after install time). It would be best to extend the stack in packages where 
> a stack extension package can have one or more custom services.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> fc1b72a 
>   ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
>   ambari-server/conf/unix/ambari.properties a88a025 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  cf7c391 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  f0928cf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java
>  557ce98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2b7fca0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  9f221d5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe9204d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
>  3721113 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
>  PRE-CREATION 
>   
> 

Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/#review136089
---



A lof of files have changed, please give me 24 hours to go over this. Thank you

- Alejandro Fernandez


On June 3, 2016, 4:54 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47656/
> ---
> 
> (Updated June 3, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth 
> Gunturi, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12885
> https://issues.apache.org/jira/browse/AMBARI-12885
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The purpose of this proposal is to facilitate adding custom services to an 
> existing stack. Ideally this would support adding and upgrading custom 
> services separately from the core services defined in the stack. In 
> particular we are looking at custom services that need to support several 
> different stacks (different distributions of Ambari). The release cycle of 
> the custom services may be different from that of the core stack; that is, a 
> custom service may be upgraded at a different rate than the core distribution 
> itself and may be upgraded multiple times within the lifespan of a single 
> release of the core distribution.
> 
> One possible approach to handling this would be dynamically extending a stack 
> (after install time). It would be best to extend the stack in packages where 
> a stack extension package can have one or more custom services.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> fc1b72a 
>   ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
>   ambari-server/conf/unix/ambari.properties a88a025 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  cf7c391 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  f0928cf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java
>  557ce98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2b7fca0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  9f221d5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe9204d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
>  3721113 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionVersionResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  99e4ccd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionDAO.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java
> 

Re: Review Request 48212: Fix files mentioned by ServicePropertiesTest on latest commits

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48212/#review136078
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 (line 32)


Why are these mandatory property attributes to cause UT failures? Shouldnt 
they have defaults if not defined?

Also how do we decide what to set these properties too? (i.e. 
on-ambari-upgrade add="false" v/s on-ambari-upgrade add="true") Can you provide 
examples? 

I am not clear on why on-ambari-upgrade delete="true" and on-stack-upgrade 
delete="false"?

I will look at your original patch to get some context and then review 
later today.


- Jayush Luniya


On June 3, 2016, 4:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48212/
> ---
> 
> (Updated June 3, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-17032
> https://issues.apache.org/jira/browse/AMBARI-17032
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /tm...
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  d0acdda 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  410a1c1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  d84207a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  4a5fbfb 
> 
> Diff: https://reviews.apache.org/r/48212/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48212: Fix files mentioned by ServicePropertiesTest on latest commits

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48212/
---

(Updated June 3, 2016, 7:55 p.m.)


Review request for Ambari and Jayush Luniya.


Bugs: AMBARI-17032
https://issues.apache.org/jira/browse/AMBARI-17032


Repository: ambari


Description
---

{code}
ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
/tm...
{code}


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 d0acdda 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 410a1c1 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 d84207a 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 4a5fbfb 

Diff: https://reviews.apache.org/r/48212/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/
---

(Updated June 3, 2016, 4:54 p.m.)


Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, 
and Yusaku Sako.


Changes
---

Re-merged over latest trunk code


Bugs: AMBARI-12885
https://issues.apache.org/jira/browse/AMBARI-12885


Repository: ambari


Description
---

The purpose of this proposal is to facilitate adding custom services to an 
existing stack. Ideally this would support adding and upgrading custom services 
separately from the core services defined in the stack. In particular we are 
looking at custom services that need to support several different stacks 
(different distributions of Ambari). The release cycle of the custom services 
may be different from that of the core stack; that is, a custom service may be 
upgraded at a different rate than the core distribution itself and may be 
upgraded multiple times within the lifespan of a single release of the core 
distribution.

One possible approach to handling this would be dynamically extending a stack 
(after install time). It would be best to extend the stack in packages where a 
stack extension package can have one or more custom services.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
fc1b72a 
  ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
  ambari-server/conf/unix/ambari.properties a88a025 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
 cf7c391 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 f0928cf 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java
 557ce98 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 2b7fca0 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 9f221d5 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 fe9204d 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
 3721113 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionVersionResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
 99e4ccd 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionDAO.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionEntity.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/stack/BaseModule.java 
ef2438f 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
 cbbdb91 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java 
65da145 
  

Review Request 48212: Fix files mentioned by ServicePropertiesTest on latest commits

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48212/
---

Review request for Ambari and Jayush Luniya.


Bugs: AMBARI-17032
https://issues.apache.org/jira/browse/AMBARI-17032


Repository: ambari


Description
---

{code}
ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
/tm...
{code}


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 d0acdda 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 410a1c1 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 d84207a 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 4a5fbfb 

Diff: https://reviews.apache.org/r/48212/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 47746: Update Moment.js to latest stable version 2.13.0

[Sangeeta Ravindran]

> On June 2, 2016, 10:34 p.m., Jaimin Jetly wrote:
> > Ship It!

Hi Jaimin,

Thank you. Can you help commit the fix. Thanks in advance.

-Sangeeta


- Sangeeta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47746/#review136003
---


On June 2, 2016, 10:16 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47746/
> ---
> 
> (Updated June 2, 2016, 10:16 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Yusaku Sako.
> 
> 
> Bugs: ambari-16017
> https://issues.apache.org/jira/browse/ambari-16017
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The latest stable version of Moment.js is 2.13.0.
> This task is for updating the version of Moment.js used in
> 
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/
> ambari-web/vendor/scripts
> 
> 
> Diffs
> -
> 
>   ambari-web/config.coffee 6ed0915 
>   ambari-web/karma.conf.js b11cf76 
>   ambari-web/vendor/scripts/moment.js fe0e19c 
>   ambari-web/vendor/scripts/moment.min.js PRE-CREATION 
>   contrib/views/slider/src/main/resources/ui/config.js 9527393 
>   contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.js 
> b40374b 
>   
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.min.js
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47746/diff/
> 
> 
> Testing
> ---
> 
> Manual testing - setting timezone in User Settings.
> Slider view - application start time.
> 
> Tests ran clean for ambari-web 
> 
> 27834 tests complete (42 seconds)
>   154 tests pending
> 
> Tests ran clean for contrib/views/slider
> Took 9506ms to run 345 tests. 345 passed, 0 failed.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/#review136067
---


Ship it!




Ship It!

- Jonathan Hurley


On June 3, 2016, 11:15 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47730/
> ---
> 
> (Updated June 3, 2016, 11:15 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Myroslav 
> Papirkovskyy, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16821
> https://issues.apache.org/jira/browse/AMBARI-16821
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The TimelineMetricsCache implementation in the metrics sink side is currently 
> a ConcurrentSkipListMap. It is better to use pre built cache libraries like 
> Guava that offer more support.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml 41ba62e 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
>  0bed7d0 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
>  18d973c 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
>  ea7f72d 
>   ambari-project/pom.xml 2fbb1e1 
>   ambari-server/pom.xml 20d3fab 
> 
> Diff: https://reviews.apache.org/r/47730/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

[Dmytro Sen]

> On Июнь 3, 2016, 2:34 п.п., Jonathan Hurley wrote:
> > ambari-metrics/ambari-metrics-common/pom.xml, line 89
> > 
> >
> > Can you explain how this works with the relocation pattern below? 
> > 
> > I get that you're relocating com.google.* classes at the bytecode level 
> > so that they they don't conflict with Guava stuff on the hadoop classpath.
> > 
> > However, this line seems to indicate that you're also packaging up the 
> > original com.google.common.* classes in your uber JAR. I would think that 
> > this is not desired since you're already relocating (shading) them.

That was new to me. Thank you for the hint.
With no 's , classes are still in the jar

META-INF/maven/com.google.guava/
META-INF/maven/com.google.guava/guava/
META-INF/maven/com.google.guava/guava/pom.properties
META-INF/maven/com.google.guava/guava/pom.xml
org/apache/hadoop/metrics2/sink/relocated/google/common/
org/apache/hadoop/metrics2/sink/relocated/google/common/annotations/
org/apache/hadoop/metrics2/sink/relocated/google/common/annotations/Beta.class
org/apache/hadoop/metrics2/sink/relocated/google/common/annotations/GwtCompatible.class
org/apache/hadoop/metrics2/sink/relocated/google/common/annotations/GwtIncompatible.class
org/apache/hadoop/metrics2/sink/relocated/google/common/annotations/VisibleForTesting.class


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/#review136064
---


On Июнь 3, 2016, 1:54 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47730/
> ---
> 
> (Updated Июнь 3, 2016, 1:54 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Myroslav 
> Papirkovskyy, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16821
> https://issues.apache.org/jira/browse/AMBARI-16821
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The TimelineMetricsCache implementation in the metrics sink side is currently 
> a ConcurrentSkipListMap. It is better to use pre built cache libraries like 
> Guava that offer more support.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml 41ba62e 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
>  0bed7d0 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
>  18d973c 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
>  ea7f72d 
>   ambari-project/pom.xml 2fbb1e1 
>   ambari-server/pom.xml 20d3fab 
> 
> Diff: https://reviews.apache.org/r/47730/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/
---

(Updated Июнь 3, 2016, 3:15 п.п.)


Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Myroslav 
Papirkovskyy, and Sid Wagle.


Bugs: AMBARI-16821
https://issues.apache.org/jira/browse/AMBARI-16821


Repository: ambari


Description
---

The TimelineMetricsCache implementation in the metrics sink side is currently a 
ConcurrentSkipListMap. It is better to use pre built cache libraries like Guava 
that offer more support.


Diffs (updated)
-

  ambari-metrics/ambari-metrics-common/pom.xml 41ba62e 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
 0bed7d0 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
 18d973c 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 ea7f72d 
  ambari-project/pom.xml 2fbb1e1 
  ambari-server/pom.xml 20d3fab 

Diff: https://reviews.apache.org/r/47730/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136065
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 3, 2016, 5:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 3, 2016, 5:38 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 48207: Agents should automatically restart if there is a memory leak

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/#review136064
---




ambari-metrics/ambari-metrics-common/pom.xml (line 89)


Can you explain how this works with the relocation pattern below? 

I get that you're relocating com.google.* classes at the bytecode level so 
that they they don't conflict with Guava stuff on the hadoop classpath.

However, this line seems to indicate that you're also packaging up the 
original com.google.common.* classes in your uber JAR. I would think that this 
is not desired since you're already relocating (shading) them.


- Jonathan Hurley


On June 3, 2016, 9:54 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47730/
> ---
> 
> (Updated June 3, 2016, 9:54 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Myroslav 
> Papirkovskyy, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16821
> https://issues.apache.org/jira/browse/AMBARI-16821
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The TimelineMetricsCache implementation in the metrics sink side is currently 
> a ConcurrentSkipListMap. It is better to use pre built cache libraries like 
> Guava that offer more support.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml 41ba62e 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
>  0bed7d0 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
>  18d973c 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
>  ea7f72d 
>   ambari-project/pom.xml 2fbb1e1 
>   ambari-server/pom.xml 20d3fab 
> 
> Diff: https://reviews.apache.org/r/47730/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 48204: HDP-UTILs Repo URL validation/save fails

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48204/#review136062
---


Ship it!




Ship It!

- Robert Levas


On June 3, 2016, 8:43 a.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48204/
> ---
> 
> (Updated June 3, 2016, 8:43 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Robert Levas.
> 
> 
> Bugs: AMBARI-17028
> https://issues.apache.org/jira/browse/AMBARI-17028
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDP-UTILS for HDP-2.5 was moved to 1.1.0.21 without our knowledge, so the API 
> only recognized updating 1.1.0.20.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml 2119413 
> 
> Diff: https://reviews.apache.org/r/48204/diff/
> 
> 
> Testing
> ---
> 
> No automated testing, it's just XML change.  Manual tested following:
> 
> - Default Install of 2.5.0.0 (make no URL changes, just "click through").
> - Install 2.5.0.0, but change the URL to the version previous
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/
---

(Updated Июнь 3, 2016, 1:52 п.п.)


Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-16821
https://issues.apache.org/jira/browse/AMBARI-16821


Repository: ambari


Description
---

The TimelineMetricsCache implementation in the metrics sink side is currently a 
ConcurrentSkipListMap. It is better to use pre built cache libraries like Guava 
that offer more support.


Diffs (updated)
-

  ambari-metrics/ambari-metrics-common/pom.xml 41ba62e 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
 0bed7d0 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
 18d973c 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 ea7f72d 
  ambari-project/pom.xml 2fbb1e1 
  ambari-server/pom.xml 20d3fab 

Diff: https://reviews.apache.org/r/47730/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 48204: HDP-UTILs Repo URL validation/save fails

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48204/#review136061
---


Ship it!




Ship It!

- Jonathan Hurley


On June 3, 2016, 8:43 a.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48204/
> ---
> 
> (Updated June 3, 2016, 8:43 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Robert Levas.
> 
> 
> Bugs: AMBARI-17028
> https://issues.apache.org/jira/browse/AMBARI-17028
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDP-UTILS for HDP-2.5 was moved to 1.1.0.21 without our knowledge, so the API 
> only recognized updating 1.1.0.20.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml 2119413 
> 
> Diff: https://reviews.apache.org/r/48204/diff/
> 
> 
> Testing
> ---
> 
> No automated testing, it's just XML change.  Manual tested following:
> 
> - Default Install of 2.5.0.0 (make no URL changes, just "click through").
> - Install 2.5.0.0, but change the URL to the version previous
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Review Request 48205: Cluster operator and ServiceAdministrator not allowed to create config group

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48205/
---

Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Myroslav 
Papirkovskyy, and Nate Cole.


Bugs: AMBARI-17029
https://issues.apache.org/jira/browse/AMBARI-17029


Repository: ambari


Description
---

Cluster operator and ServiceAdministrator are not allowed to create a config 
group even though it should be.
It is visible in the UI but creating it gives a 403 error in the backend and 
the UI goes for a toss.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
 6fe29db 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
 eeb1a8b 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 d56ed44 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
 2913cf5 
  
ambari-server/src/test/java/org/apache/ambari/server/security/TestAuthenticationFactory.java
 4301bf8 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java
 ff47ac2 

Diff: https://reviews.apache.org/r/48205/diff/


Testing
---

Manually tested

#Local test results: 
Unrelated unit test failures:
```
Results :

Tests in error:
  StackManagerTest.testMetricsLoaded:669 » Ambari File 
/Users/rlevas/git/ambari/...
  StackManagerTest.testServicesWithLogsearchRoleCommandOrder:820 » Ambari File 
/...
  StackManagerTest.testServicesWithRangerPluginRoleCommandOrder:713 » Ambari 
Fil...
  ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
/Us...

Tests run: 4458, Failures: 0, Errors: 4, Skipped: 34
```

#Jenkins test results: PENDING


Thanks,

Robert Levas

Review Request 48204: HDP-UTILs Repo URL validation/save fails

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48204/
---

Review request for Ambari, Jonathan Hurley and Robert Levas.


Bugs: AMBARI-17028
https://issues.apache.org/jira/browse/AMBARI-17028


Repository: ambari


Description
---

HDP-UTILS for HDP-2.5 was moved to 1.1.0.21 without our knowledge, so the API 
only recognized updating 1.1.0.20.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml 2119413 

Diff: https://reviews.apache.org/r/48204/diff/


Testing
---

No automated testing, it's just XML change.  Manual tested following:

- Default Install of 2.5.0.0 (make no URL changes, just "click through").
- Install 2.5.0.0, but change the URL to the version previous


Thanks,

Nate Cole

Re: Review Request 48162: 16171 Addendum2 for stackadvisor with Phoenix Query Server kerberos configuration

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48162/#review136059
---


Ship it!




Ship It!

- Jonathan Hurley


On June 2, 2016, 12:38 p.m., Josh Elser wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48162/
> ---
> 
> (Updated June 2, 2016, 12:38 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-16171
> https://issues.apache.org/jira/browse/AMBARI-16171
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix an issue where unsubstituted variables are being left in core-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  f887f92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 5cfe42d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 413a2f7 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> d61de7b 
> 
> Diff: https://reviews.apache.org/r/48162/diff/
> 
> 
> Testing
> ---
> 
> Deployed 2.4 with Kerberos and no PQS
> Deployed 2.4 with Kerberos and PQS
> (correct configs for both)
> Python UTs are passing locally
> java UTs still running locally
> 
> 
> Thanks,
> 
> Josh Elser
> 
>

Re: Review Request 48181: Service admin and cluster operator can't modify service configs through API

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48181/#review136058
---


Ship it!




Ship It!

- Nate Cole


On June 2, 2016, 2:34 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48181/
> ---
> 
> (Updated June 2, 2016, 2:34 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-17014
> https://issues.apache.org/jira/browse/AMBARI-17014
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Using a serviceoperator user, trying to modify config using the 2 step 
> modification process :
> 
> Request type : POST
> Request URL : /api/v1/clusters/cl1/configurations
> Auth : serviceadminuser/password
> Request Body :
> ```
> {"type":"ams-env","tag":"version146474298002","properties":{"ambari_metrics_user":"ams","metrics_monitor_log_dir":"/grid/0/log/metric_monitor_updated","metrics_collector_heapsize":"512","metrics_collector_pid_dir":"/var/run/ambari-metrics-collector","metrics_collector_log_dir":"/grid/0/log/metric_collector","metrics_monitor_pid_dir":"/var/run/ambari-metrics-monitor","content":"\n#
>  Set environment variables here.\n\n# The java implementation to use. Java 
> 1.6 required.\nexport JAVA_HOME\u003d{{java64_home}}\n\n# Collector Log 
> directory for log4j\nexport 
> AMS_COLLECTOR_LOG_DIR\u003d{{ams_collector_log_dir}}\n\n# Monitor Log 
> directory for outfile\nexport 
> AMS_MONITOR_LOG_DIR\u003d{{ams_monitor_log_dir}}\n\n# Collector pid 
> directory\nexport AMS_COLLECTOR_PID_DIR\u003d{{ams_collector_pid_dir}}\n\n# 
> Monitor pid directory\nexport 
> AMS_MONITOR_PID_DIR\u003d{{ams_monitor_pid_dir}}\n\n# AMS HBase pid 
> directory\nexport AMS_HBASE_PID_DIR\u003d{{hbase_pid_dir}}\n\n# AMS Collector 
> heapsize\nexport A
 MS_COLLECTOR_HEAPSIZE\u003d{{metrics_collector_heapsize}}\n\n# HBase 
normalizer enabled\nexport 
AMS_HBASE_NORMALIZER_ENABLED\u003d{{ams_hbase_normalizer_enabled}}\n\n# HBase 
compaction policy enabled\nexport 
AMS_HBASE_FIFO_COMPACTION_ENABLED\u003d{{ams_hbase_fifo_compaction_enabled}}\n\n#
 HBase Tables Initialization check enabled\nexport 
AMS_HBASE_INIT_CHECK_ENABLED\u003d{{ams_hbase_init_check_enabled}}\n\n# AMS 
Collector options\nexport 
AMS_COLLECTOR_OPTS\u003d\"-Djava.library.path\u003d/usr/lib/ams-hbase/lib/hadoop-native\"\n{%
 if security_enabled %}\nexport AMS_COLLECTOR_OPTS\u003d\"$AMS_COLLECTOR_OPTS 
-Djava.security.auth.login.config\u003d{{ams_collector_jaas_config_file}}\"\n{% 
endif %}\n\n# AMS Collector GC options\nexport 
AMS_COLLECTOR_GC_OPTS\u003d\"-XX:+UseConcMarkSweepGC -verbose:gc 
-XX:+PrintGCDetails -XX:+PrintGCDateStamps 
-Xloggc:{{ams_collector_log_dir}}/collector-gc.log-`date 
+\u0027%Y%m%d%H%M\u0027`\"\nexport 
AMS_COLLECTOR_OPTS\u003d\"$AMS_COLLECTOR_OPTS $AMS_COLLEC
 TOR_GC_OPTS\"\n\n"}}
> ```
> 
> Request response : 
> ```
> {
>   "status": 403,
>   "message": "You do not have permissions to access this resource."
> }
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
>  eeb1a8b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java
>  ff47ac2 
> 
> Diff: https://reviews.apache.org/r/48181/diff/
> 
> 
> Testing
> ---
> 
> Manaually tested
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 48162: 16171 Addendum2 for stackadvisor with Phoenix Query Server kerberos configuration

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48162/#review136057
---


Ship it!





ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
209)


%s % syntax is not the standard anymore, use "".format(...)


- Nate Cole


On June 2, 2016, 12:38 p.m., Josh Elser wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48162/
> ---
> 
> (Updated June 2, 2016, 12:38 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-16171
> https://issues.apache.org/jira/browse/AMBARI-16171
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix an issue where unsubstituted variables are being left in core-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  f887f92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 5cfe42d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 413a2f7 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> d61de7b 
> 
> Diff: https://reviews.apache.org/r/48162/diff/
> 
> 
> Testing
> ---
> 
> Deployed 2.4 with Kerberos and no PQS
> Deployed 2.4 with Kerberos and PQS
> (correct configs for both)
> Python UTs are passing locally
> java UTs still running locally
> 
> 
> Thanks,
> 
> Josh Elser
> 
>

Re: Review Request 48173: Fix logfeeder filter name of logsearch

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48173/
---

(Updated June 3, 2016, 10:53 a.m.)


Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert Nettleton, 
and Sumit Mohanty.


Changes
---

rebase changes for trunk/branch-2.4


Bugs: AMBARI-17013
https://issues.apache.org/jira/browse/AMBARI-17013


Repository: ambari


Description
---

small fix to use the right name for logfeeder filter on logsearch 
side.(AMBARI-16760 changed that wrongly)


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
 f9f81b4 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/default.properties 
7c2f858 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
 eaa1106 

Diff: https://reviews.apache.org/r/48173/diff/


Testing
---

FT: with 4 node cluster


Thanks,

Oliver Szabo

Re: Review Request 48121: YARN default configs are invalid

[Dmytro Sen]

> On Июнь 2, 2016, 4:55 п.п., Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> >  line 337
> > 
> >
> > This code looks fine to me, but are we sure that there are no 
> > properties in an HDP cluster that may consider trailing spaces to be a 
> > valid property value? 
> > 
> > It's probably fine, but just wanted to make sure we double-check this. 
> > 
> > Thanks.

Not all the properies should be trimmed. I'll implement trimming in a separate 
jira.


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48121/#review135962
---


On Июнь 3, 2016, 10:43 д.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48121/
> ---
> 
> (Updated Июнь 3, 2016, 10:43 д.п.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Robert Nettleton, Sebastian 
> Toader, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16979
> https://issues.apache.org/jira/browse/AMBARI-16979
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The default value of yarn.resourcemanager.zk-acl is set to 
> "world:anyone:rwcda ". The extra space is causing the yarn config to be 
> considered invalid
> The default value of scheduler shows empty.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  858f375 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml
>  9777ee4 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/configuration/yarn-site.xml
>  c8a588c 
> 
> Diff: https://reviews.apache.org/r/48121/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 48121: YARN default configs are invalid

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48121/
---

(Updated Июнь 3, 2016, 10:43 д.п.)


Review request for Ambari, Andrew Onischuk, Robert Nettleton, Sebastian Toader, 
and Vitalyi Brodetskyi.


Bugs: AMBARI-16979
https://issues.apache.org/jira/browse/AMBARI-16979


Repository: ambari


Description
---

The default value of yarn.resourcemanager.zk-acl is set to "world:anyone:rwcda 
". The extra space is causing the yarn config to be considered invalid
The default value of scheduler shows empty.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
 858f375 
  
ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/YARN/configuration/yarn-site.xml
 9777ee4 
  
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/configuration/yarn-site.xml
 c8a588c 

Diff: https://reviews.apache.org/r/48121/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 48202: NPE during EU at Update Target Stack step

[Dmytro Grinenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48202/#review136053
---


Ship it!




Ship It!

- Dmytro Grinenko


On June 3, 2016, 10:05 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48202/
> ---
> 
> (Updated June 3, 2016, 10:05 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Dmytro Grinenko.
> 
> 
> Bugs: AMBARI-17026
> https://issues.apache.org/jira/browse/AMBARI-17026
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.4.2.0 with Ambari 2.4.0.0-630
> # Start EU to 2.5.0.0-639
> 
> *Result*
> NPE at "Update Target Stack"
> {code}
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.controller.internal.UpgradeResourceProvider.applyStackAndProcessConfigurations(UpgradeResourceProvider.java:1114)
> at 
> org.apache.ambari.server.serveraction.upgrades.UpdateDesiredStackAction.updateDesiredStack(UpdateDesiredStackAction.java:180)
> at 
> org.apache.ambari.server.serveraction.upgrades.UpdateDesiredStackAction.execute(UpdateDesiredStackAction.java:121)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 6ea7983 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  e55058f 
> 
> Diff: https://reviews.apache.org/r/48202/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48202: NPE during EU at Update Target Stack step

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48202/#review136052
---


Ship it!




Ship It!

- Andrew Onischuk


On June 3, 2016, 10:05 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48202/
> ---
> 
> (Updated June 3, 2016, 10:05 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Dmytro Grinenko.
> 
> 
> Bugs: AMBARI-17026
> https://issues.apache.org/jira/browse/AMBARI-17026
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.4.2.0 with Ambari 2.4.0.0-630
> # Start EU to 2.5.0.0-639
> 
> *Result*
> NPE at "Update Target Stack"
> {code}
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.controller.internal.UpgradeResourceProvider.applyStackAndProcessConfigurations(UpgradeResourceProvider.java:1114)
> at 
> org.apache.ambari.server.serveraction.upgrades.UpdateDesiredStackAction.updateDesiredStack(UpdateDesiredStackAction.java:180)
> at 
> org.apache.ambari.server.serveraction.upgrades.UpdateDesiredStackAction.execute(UpdateDesiredStackAction.java:121)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 6ea7983 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  e55058f 
> 
> Diff: https://reviews.apache.org/r/48202/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 48202: NPE during EU at Update Target Stack step

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48202/
---

Review request for Ambari, Andrew Onischuk and Dmytro Grinenko.


Bugs: AMBARI-17026
https://issues.apache.org/jira/browse/AMBARI-17026


Repository: ambari


Description
---

*Steps*
# Deploy HDP-2.4.2.0 with Ambari 2.4.0.0-630
# Start EU to 2.5.0.0-639

*Result*
NPE at "Update Target Stack"
{code}
java.lang.NullPointerException
at 
org.apache.ambari.server.controller.internal.UpgradeResourceProvider.applyStackAndProcessConfigurations(UpgradeResourceProvider.java:1114)
at 
org.apache.ambari.server.serveraction.upgrades.UpdateDesiredStackAction.updateDesiredStack(UpdateDesiredStackAction.java:180)
at 
org.apache.ambari.server.serveraction.upgrades.UpdateDesiredStackAction.execute(UpdateDesiredStackAction.java:121)
at 
org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
at 
org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
at java.lang.Thread.run(Thread.java:745)
{code}


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
6ea7983 
  
ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
 e55058f 

Diff: https://reviews.apache.org/r/48202/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 48168: Check new jdbc functionality for SQLA and provide additional testing

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48168/#review136051
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 2, 2016, 9:04 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48168/
> ---
> 
> (Updated June 2, 2016, 9:04 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro 
> Sen.
> 
> 
> Bugs: AMBARI-17007
> https://issues.apache.org/jira/browse/AMBARI-17007
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> .
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py
>  9e298a4 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py
>  908c726 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  7657dbc 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
>  ffe1783 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  b76bc89 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
>  e446c43 
> 
> Diff: https://reviews.apache.org/r/48168/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 48194: Update name of PXF component to PXF Agent

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48194/#review136045
---


Ship it!




Ship It!

- Matt


On June 2, 2016, 6:10 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48194/
> ---
> 
> (Updated June 2, 2016, 6:10 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-17024
> https://issues.apache.org/jira/browse/AMBARI-17024
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update name of PXF component to PXF Agent
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
> ba1c58e 
> 
> Diff: https://reviews.apache.org/r/48194/diff/
> 
> 
> Testing
> ---
> 
> yes.
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>