Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136108
---
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
(line 1175)
<https://reviews.apache.org/r/48221/#comment201118>
one day we'll support multi-cluster, so maybe this should combine cluster
name + component
- Nate Cole
On June 3, 2016, 4:56 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
>
> (Updated June 3, 2016, 4:56 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
>
>
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When retrieving JMX properties on large clusters, if there are JMX endpoints
> which are unreachable, the URI protocol value is never cached. As a result,
> multiple attempts are made to get the JMX protocol. Normally, this would be
> fine, except that retrieving the configuration is using the
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
>
> {code}
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
> at
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
> at
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
> at
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
> at
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
> at
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
> at
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
> at
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
> at
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
> at
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
> at
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
> at
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
> at
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
> at
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
> at
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
> at
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
> at
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
> at
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
> at
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
> at
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
> at
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
> at
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
> at
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.cal
Re: Review Request 48221: Getting JMX Protocol Values On Large Cluster Takes Too Long
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48221/#review136112
---
Ship it!
Ship It!
- Nate Cole
On June 3, 2016, 5:30 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48221/
> ---
>
> (Updated June 3, 2016, 5:30 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
>
>
> Bugs: AMBARI-17036
> https://issues.apache.org/jira/browse/AMBARI-17036
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When retrieving JMX properties on large clusters, if there are JMX endpoints
> which are unreachable, the URI protocol value is never cached. As a result,
> multiple attempts are made to get the JMX protocol. Normally, this would be
> fine, except that retrieving the configuration is using the
> {{ClusterResourceProvider}}, which in turn, builds a {{ClusterHealthReport}}.
>
> {code}
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO.findByPK(ServiceDesiredStateDAO.java:41)
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.CGLIB$findByPK$1()
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf$$FastClassByGuice$$9cad416f.invoke()
> at
> com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
> at
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:43)
> at
> com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
> at
> com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
> at
> org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO$$EnhancerByGuice$$d1ec71cf.findByPK()
> at
> org.apache.ambari.server.state.ServiceImpl.getServiceDesiredStateEntity(ServiceImpl.java:759)
> at
> org.apache.ambari.server.state.ServiceImpl.getMaintenanceState(ServiceImpl.java:732)
> at
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:216)
> at
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:180)
> at
> org.apache.ambari.server.controller.MaintenanceStateHelper.getEffectiveState(MaintenanceStateHelper.java:194)
> at
> org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:3136)
> at
> org.apache.ambari.server.state.cluster.ClusterImpl.convertToResponse(ClusterImpl.java:2084)
> at
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:950)
> at
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.getClusters(AmbariManagementControllerImpl.java:3082)
> at
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:202)
> at
> org.apache.ambari.server.controller.internal.ClusterResourceProvider$1.invoke(ClusterResourceProvider.java:199)
> at
> org.apache.ambari.server.controller.internal.AbstractResourceProvider.getResources(AbstractResourceProvider.java:302)
> at
> org.apache.ambari.server.controller.internal.ClusterResourceProvider.getResources(ClusterResourceProvider.java:199)
> at
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getDesiredConfigVersion(AbstractProviderModule.java:954)
> at
> org.apache.ambari.server.controller.internal.AbstractProviderModule.getJMXProtocol(AbstractProviderModule.java:1154)
> at
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.getJMXProtocol(JMXPropertyProvider.java:402)
> at
> org.apache.ambari.server.controller.jmx.JMXPropertyProvider.populateResource(JMXPropertyProvider.java:221)
> at
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:222)
> at
> org.apache.ambari.server.controller.metrics.ThreadPoolEnabledPropertyProvider$1.call(ThreadPoolEnabledPropertyProvider.java:219)
> at java.util.concurrent.FutureTask.run(FutureTas
Re: Review Request 48403: Fixed implementation of on-ambari-upgrade support. Patch 1 - change validation rules and available fields
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48403/#review136659 --- ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java (lines 53 - 54) <https://reviews.apache.org/r/48403/#comment201756> Is this being removed to focus only on Ambari Upgrade - Nate Cole On June 8, 2016, 6:06 a.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48403/ > --- > > (Updated June 8, 2016, 6:06 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush > Luniya, Nate Cole, and Sumit Mohanty. > > > Bugs: AMBARI-17112 > https://issues.apache.org/jira/browse/AMBARI-17112 > > > Repository: ambari > > > Description > --- > > We decided to split changes to stack config xmls and processing of them into > few patches. > Current patch changes validation rules and available fields > > Splitted PropertyUpgradeBehavior into 2 classes. > PropertyAmbariUpgradeBehavior - defines behavior for ambari upgrade. > on-ambari-upgrade now has the same 3 attributes, but all are optional. > Default value for add is true, false for others. > > PropertyStackUpgradeBehavior - defines behavior for stack upgrade. > on-stack-upgrade now has one attribute - add, and it is required. > > Modified property update script and xsd schema to satisfy new requirements. > > If patch looks good, I'll modify all stack configs to comply with it and > commit both patches at the same time. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > fba2daa > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java > de2e342 > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java > de2e342 > ambari-server/src/main/resources/configuration-schema.xsd d49cbf8 > > ambari-server/src/main/resources/scripts/configurations-set-default-update-policy.sh > 930f778 > > ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java > 9a3d195 > > Diff: https://reviews.apache.org/r/48403/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 48436: Ambari Stale Alert Triggers For Server-Side Performance Alert
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48436/#review136800
---
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/alerts/StaleAlertRunnable.java
(lines 76 - 81)
<https://reviews.apache.org/r/48436/#comment201885>
Should this be a parameter of the alert definition?
- Nate Cole
On June 8, 2016, 2:24 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48436/
> ---
>
> (Updated June 8, 2016, 2:24 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
>
>
> Bugs: AMBARI-17127
> https://issues.apache.org/jira/browse/AMBARI-17127
>
>
> Repository: ambari
>
>
> Description
> ---
>
> If the Ambari Server is restarted after being down for several minutes, the
> "Ambari Server Performance" alert will trigger as being stale. This is
> because the {{StaleAlertRunnable}} is not checking to ensure that Ambari has
> been up and running long enough to have run the alert.
>
> The {{StaleAlertRunnable}} should verify that the uptime of Ambari is greater
> than staleness interval.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/alerts/StaleAlertRunnable.java
> cf12bbf
>
> ambari-server/src/test/java/org/apache/ambari/server/alerts/StaleAlertRunnableTest.java
> 9a9d989
>
> Diff: https://reviews.apache.org/r/48436/diff/
>
>
> Testing
> ---
>
> Failure not related to patch.
>
> Failed tests:
> ViewPrivilegeEventCreatorTest.putTest:85 expected:<...missions(
> Permission[1:
> Users: testuser
> Groups: testgroup
> Permission2:
> Users: testuser2])> but was:<...missions(
> Permission[2:
> Users: testuser2
> Permission1:
> Users: testuser
> Groups: testgroup])>
>
> Tests run: 4468, Failures: 1, Errors: 0, Skipped: 34
>
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 34:11 min
> [INFO] Finished at: 2016-06-08T01:10:29-04:00
> [INFO] Final Memory: 38M/645M
> [INFO]
>
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 48403: Fixed implementation of on-ambari-upgrade support. Patch 1 - change validation rules and available fields
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48403/#review136808 --- Ship it! Ship It! - Nate Cole On June 9, 2016, 9:12 a.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48403/ > --- > > (Updated June 9, 2016, 9:12 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jonathan > Hurley, Jayush Luniya, Nate Cole, and Sumit Mohanty. > > > Bugs: AMBARI-17112 > https://issues.apache.org/jira/browse/AMBARI-17112 > > > Repository: ambari > > > Description > --- > > We decided to split changes to stack config xmls and processing of them into > few patches. > Current patch changes validation rules and available fields > > Splitted PropertyUpgradeBehavior into 2 classes. > PropertyAmbariUpgradeBehavior - defines behavior for ambari upgrade. > on-ambari-upgrade now has the same 3 attributes, but all are optional. > Default value for add is true, false for others. > > PropertyStackUpgradeBehavior - defines behavior for stack upgrade. > on-stack-upgrade now has one attribute - add, and it is required. > > Modified property update script and xsd schema to satisfy new requirements. > > If patch looks good, I'll modify all stack configs to comply with it and > commit both patches at the same time. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > fba2daa > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java > de2e342 > > ambari-server/src/main/resources/scripts/configurations-set-default-update-policy.sh > 930f778 > > ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java > 9a3d195 > > Diff: https://reviews.apache.org/r/48403/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 48287: RU failed because of old service was not stopped
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136347
---
Ship it!
Ship It!
- Nate Cole
On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
>
> (Updated June 6, 2016, 1:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
>
>
> Error message:
>
> {code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
> line 211, in
> HiveServer().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 257, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process
> keep running.
> {code}
>
>
> Diffs
> -
>
> ambari-agent/src/test/python/resource_management/TestScript.py adb8501
>
> ambari-common/src/main/python/resource_management/libraries/script/script.py
> 61759cc
>
> Diff: https://reviews.apache.org/r/48287/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 48287: RU failed because of old service was not stopped
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136319
---
I only see two changes here, and it's just logging. Was there some missed
files?
- Nate Cole
On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
>
> (Updated June 6, 2016, 1:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
>
>
> Error message:
>
> {code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
> line 211, in
> HiveServer().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 257, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process
> keep running.
> {code}
>
>
> Diffs
> -
>
> ambari-agent/src/test/python/resource_management/TestScript.py adb8501
>
> ambari-common/src/main/python/resource_management/libraries/script/script.py
> 61759cc
>
> Diff: https://reviews.apache.org/r/48287/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Review Request 48292: VDF: exception when trying to register -> add versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48292/ --- Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Jonathan Hurley. Bugs: AMBARI-17071 https://issues.apache.org/jira/browse/AMBARI-17071 Repository: ambari Description --- Only verify duplicate repo URLs if Ambari is managing the repos for the desired repo_version. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java 57fb115 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java a61af95 Diff: https://reviews.apache.org/r/48292/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
Re: Review Request 48258: Fix description of SERVICE.ADD_DELETE_SERVICES permission
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48258/#review136359 --- Ship it! Ship It! - Nate Cole On June 6, 2016, 3:54 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48258/ > --- > > (Updated June 6, 2016, 3:54 p.m.) > > > Review request for Ambari, Denys Buzhor, Jonathan Robie, Nate Cole, and > Yusaku Sako. > > > Bugs: AMBARI-17043 > https://issues.apache.org/jira/browse/AMBARI-17043 > > > Repository: ambari > > > Description > --- > > The description of the SERVICE.ADD_DELETE_SERVICES permission currently reads > > ``` > Add Service to cluster > ``` > > This should be changed to > > ``` > Add/delete services > ``` > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog230.java > be9c2e2 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 01322b2 > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 940542d > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql eb2b349 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql de8c2e6 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 0a8d6c9 > ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql > 4b65a69 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 5ef07d0 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 0b5f3b8 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > 670200c > > Diff: https://reviews.apache.org/r/48258/diff/ > > > Testing > --- > > Manually tested new cluster and upgrade. > > > Thanks, > > Robert Levas > >
Re: Review Request 48292: VDF: exception when trying to register -> add versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48292/ --- (Updated June 6, 2016, 5:34 p.m.) Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Jonathan Hurley. Bugs: AMBARI-17071 https://issues.apache.org/jira/browse/AMBARI-17071 Repository: ambari Description --- Only verify duplicate repo URLs if Ambari is managing the repos for the desired repo_version. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java 57fb115 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java a61af95 Diff: https://reviews.apache.org/r/48292/diff/ Testing (updated) --- Manual. Automated: Tests run: 4454, Failures: 0, Errors: 0, Skipped: 34 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 36:27.253s [INFO] Finished at: Mon Jun 06 17:26:34 EDT 2016 [INFO] Final Memory: 35M/569M [INFO] Thanks, Nate Cole
Re: Review Request 48557: Fixed implementation of on-ambari-upgrade support. Patch 2: add logic for ambari-upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48557/#review137043 --- There should be some java tests for this new functionality. - Nate Cole On June 10, 2016, 1:10 p.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48557/ > --- > > (Updated June 10, 2016, 1:10 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, > and Sumit Mohanty. > > > Bugs: AMBARI-17112 > https://issues.apache.org/jira/browse/AMBARI-17112 > > > Repository: ambari > > > Description > --- > > Patch implements logic for on-ambari-upgrade attributes: add, update, delete > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > c570ab3 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 3ee8bba > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-site.xml > 6793e4e > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > fd68ae0 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-startup.properties.xml > aacf10a > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-site.xml > 4e56084 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/core-site.xml > 7f67b8a > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml > 274163e > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml > b0f36e7 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml > 444b8b4 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml > cfdd989 > > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml > 7f474f6 > > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml > f3a6bcf > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml > 47c900e > > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml > a309fa4 > > ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-site.xml > 2d0047c > > ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/ranger-env.xml > 5082277 > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml > 121a797 > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml > ae56f8b > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml > 4317cfa > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml > 7eb78e5 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml > a747dde > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 35910ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml > 6eb312f > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 9870b8b > > ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml > 633cbda > > ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml > daacdee > > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml > 21cd096 > > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml > a8f1584 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml > d7c87e9 > > ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml > 7aa18cc > > ambari-server/src/main/resources/common-services
Re: Review Request 48549: AMBARI-17165 Handle Java patches execution during Ranger upgrade
> On June 10, 2016, 1:40 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml, > > line 889 > > <https://reviews.apache.org/r/48549/diff/1/?file=1414725#file1414725line889> > > > > hosts="all" is the default, you can remove it. > > sequential="true" isn't needed since it is only useful when two > > consecutive tasks need to run one after the other even though they are on > > different hosts. > > > > Why is there a stop and start in post-upgrade? > > What are you trying to achieve here? > > > > Pre-upgrade is any steps to execute before restarting the component, > > Post-upgrade is after restarting. So if there are any other stop-start > > steps, it means the component is being restarted twice! I agree with Alejandro here, it looks like you're stopping all admins, java-patching only one, then starting them all. They should all need patching, no? And if that's the case that can all be handled on the python side with nothing but restart for the component in the UP. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48549/#review137042 --- On June 10, 2016, 8:47 a.m., Mugdha Varadkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48549/ > --- > > (Updated June 10, 2016, 8:47 a.m.) > > > Review request for Ambari, Gautam Borad, Jonathan Hurley, Nate Cole, Srimanth > Gunturi, and Velmurugan Periasamy. > > > Bugs: AMBARI-17165 > https://issues.apache.org/jira/browse/AMBARI-17165 > > > Repository: ambari > > > Description > --- > > Revisiting execution of java patches during upgrade for Ranger Service > > > Diffs > - > > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml > 4fd5801 > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml > 272a3cc > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml > b0cff68 > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml > 0b72254 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml > 111b432 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml > 9365646 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml > f40f760 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml > 712241b > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml > 4187d64 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml > ea5ff5a > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml > e83b54b > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml > 7fb03dc > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml > 4065e87 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml > 7f988e3 > > ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml > 460e6b3 > ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml > 6ce4c81 > > Diff: https://reviews.apache.org/r/48549/diff/ > > > Testing > --- > > Tested Ranger upgrade from stack 2.4 to 2.5 > > > Thanks, > > Mugdha Varadkar > >
Review Request 48562: Allow option to skip duplicate URL checking when creating VDF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48562/ --- Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-17173 https://issues.apache.org/jira/browse/AMBARI-17173 Repository: ambari Description --- When creating a VDF, it may be desirable to skip duplicate base URL checking. This is implemented as a Create Directive. Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java 30afa69 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java fae279d ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java c18d722 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java e0ff2b3 Diff: https://reviews.apache.org/r/48562/diff/ Testing --- Manual. Automated pending Thanks, Nate Cole
Re: Review Request 48562: Allow option to skip duplicate URL checking when creating VDF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48562/ --- (Updated June 10, 2016, 3:14 p.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-17173 https://issues.apache.org/jira/browse/AMBARI-17173 Repository: ambari Description --- When creating a VDF, it may be desirable to skip duplicate base URL checking. This is implemented as a Create Directive. Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java 30afa69 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java fae279d ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java c18d722 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java e0ff2b3 Diff: https://reviews.apache.org/r/48562/diff/ Testing (updated) --- Manual. Automated: Tests run: 4466, Failures: 0, Errors: 0, Skipped: 34 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 42:39.911s [INFO] Finished at: Fri Jun 10 15:09:35 EDT 2016 [INFO] Final Memory: 35M/661M [INFO] Thanks, Nate Cole
Re: Review Request 48498: Duplicate key in database exception during version registration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48498/#review136835 --- ambari-agent/src/main/python/ambari_agent/NetUtil.py (line 64) <https://reviews.apache.org/r/48498/#comment201921> This was an annoyance as I tried to run an agent that was upgraded after I changed ambari.ini. - Nate Cole On June 9, 2016, 12:49 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48498/ > --- > > (Updated June 9, 2016, 12:49 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. > > > Bugs: AMBARI-17130 > https://issues.apache.org/jira/browse/AMBARI-17130 > > > Repository: ambari > > > Description > --- > > UI needs to be able to specify a display_name when creating a Version > Definition. This is to avoid a name collision when installing default stack > repos. > > In addition, add a validation object to the dry_run endpoint to catch errors > without POSTing "for real" > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/NetUtil.py 79181f1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java > 049c4a7 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java > 75427ff > > Diff: https://reviews.apache.org/r/48498/diff/ > > > Testing > --- > > Manual testing. Automated (failure not related to patch): > > Failed tests: > ViewPrivilegeEventCreatorTest.putTest:85 expected:<...missions( > Permission[1: > Users: testuser > Groups: testgroup > Permission2: > Users: testuser2])> but was:<...missions( > Permission[2: > Users: testuser2 > Permission1: > Users: testuser > Groups: testgroup])> > > Tests run: 4462, Failures: 1, Errors: 0, Skipped: 34 > > [INFO] > > [INFO] BUILD FAILURE > [INFO] > ---- > [INFO] Total time: 42:33.638s > [INFO] Finished at: Thu Jun 09 12:40:14 EDT 2016 > [INFO] Final Memory: 33M/689M > [INFO] > > > > Thanks, > > Nate Cole > >
Review Request 48498: Duplicate key in database exception during version registration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48498/ --- Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-17130 https://issues.apache.org/jira/browse/AMBARI-17130 Repository: ambari Description --- UI needs to be able to specify a display_name when creating a Version Definition. This is to avoid a name collision when installing default stack repos. In addition, add a validation object to the dry_run endpoint to catch errors without POSTing "for real" Diffs - ambari-agent/src/main/python/ambari_agent/NetUtil.py 79181f1 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 049c4a7 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java 75427ff Diff: https://reviews.apache.org/r/48498/diff/ Testing --- Manual testing. Automated (failure not related to patch): Failed tests: ViewPrivilegeEventCreatorTest.putTest:85 expected:<...missions( Permission[1: Users: testuser Groups: testgroup Permission2: Users: testuser2])> but was:<...missions( Permission[2: Users: testuser2 Permission1: Users: testuser Groups: testgroup])> Tests run: 4462, Failures: 1, Errors: 0, Skipped: 34 [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 42:33.638s [INFO] Finished at: Thu Jun 09 12:40:14 EDT 2016 [INFO] Final Memory: 33M/689M [INFO] ---- Thanks, Nate Cole
Re: Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48234/#review136145 --- Ship it! Ship It! - Nate Cole On June 3, 2016, 6:47 p.m., Alejandro Fernandez wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48234/ > --- > > (Updated June 3, 2016, 6:47 p.m.) > > > Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe. > > > Bugs: AMBARI-17037 > https://issues.apache.org/jira/browse/AMBARI-17037 > > > Repository: ambari > > > Description > --- > > Fresh installation and EU/RU of Falcon is failing. > In HDP 2.4, Falcon used the data mirroring folder. > In HDP 2.5, data mirroring is no longer used and instead uses the extensions > folder that must be uploaded to HDFS. > > This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions > folder to HDFS > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/constants.py > 34d1a1a > > ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py > 4c13aaa > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py > 5e25325 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py > 7c2cdf4 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py > 8c2ad8e > > ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json > 8b669e8 > ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 > > Diff: https://reviews.apache.org/r/48234/diff/ > > > Testing > --- > > Python unit tests passed. > > -- > Total run:1052 > Total errors:0 > Total failures:0 > OK > > Express Upgrade is currently broken, so will commit this patch and retest > once it is stable again in Hadoop Core. > > > Thanks, > > Alejandro Fernandez > >
Re: Review Request 48205: Cluster operator and ServiceAdministrator not allowed to create config group
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48205/#review136118 --- Ship it! Ship It! - Nate Cole On June 3, 2016, 9:56 a.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48205/ > --- > > (Updated June 3, 2016, 9:56 a.m.) > > > Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Myroslav > Papirkovskyy, and Nate Cole. > > > Bugs: AMBARI-17029 > https://issues.apache.org/jira/browse/AMBARI-17029 > > > Repository: ambari > > > Description > --- > > Cluster operator and ServiceAdministrator are not allowed to create a config > group even though it should be. > It is visible in the UI but creating it gives a 403 error in the backend and > the UI goes for a toss. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java > 6fe29db > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java > 2b9f304 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java > d56ed44 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java > 2913cf5 > > ambari-server/src/test/java/org/apache/ambari/server/security/TestAuthenticationFactory.java > 4301bf8 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java > 96b2cfb > > Diff: https://reviews.apache.org/r/48205/diff/ > > > Testing > --- > > Manually tested > > #Local test results: > Unrelated unit test failures: > ``` > Results : > > Tests in error: > StackManagerTest.testMetricsLoaded:669 » Ambari File > /Users/rlevas/git/ambari/... > StackManagerTest.testServicesWithLogsearchRoleCommandOrder:820 » Ambari > File /... > StackManagerTest.testServicesWithRangerPluginRoleCommandOrder:713 » Ambari > Fil... > ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File > /Us... > > Tests run: 4458, Failures: 0, Errors: 4, Skipped: 34 > ``` > > #Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 48181: Service admin and cluster operator can't modify service configs through API
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48181/#review136058
---
Ship it!
Ship It!
- Nate Cole
On June 2, 2016, 2:34 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48181/
> ---
>
> (Updated June 2, 2016, 2:34 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Nate
> Cole.
>
>
> Bugs: AMBARI-17014
> https://issues.apache.org/jira/browse/AMBARI-17014
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Using a serviceoperator user, trying to modify config using the 2 step
> modification process :
>
> Request type : POST
> Request URL : /api/v1/clusters/cl1/configurations
> Auth : serviceadminuser/password
> Request Body :
> ```
> {"type":"ams-env","tag":"version146474298002","properties":{"ambari_metrics_user":"ams","metrics_monitor_log_dir":"/grid/0/log/metric_monitor_updated","metrics_collector_heapsize":"512","metrics_collector_pid_dir":"/var/run/ambari-metrics-collector","metrics_collector_log_dir":"/grid/0/log/metric_collector","metrics_monitor_pid_dir":"/var/run/ambari-metrics-monitor","content":"\n#
> Set environment variables here.\n\n# The java implementation to use. Java
> 1.6 required.\nexport JAVA_HOME\u003d{{java64_home}}\n\n# Collector Log
> directory for log4j\nexport
> AMS_COLLECTOR_LOG_DIR\u003d{{ams_collector_log_dir}}\n\n# Monitor Log
> directory for outfile\nexport
> AMS_MONITOR_LOG_DIR\u003d{{ams_monitor_log_dir}}\n\n# Collector pid
> directory\nexport AMS_COLLECTOR_PID_DIR\u003d{{ams_collector_pid_dir}}\n\n#
> Monitor pid directory\nexport
> AMS_MONITOR_PID_DIR\u003d{{ams_monitor_pid_dir}}\n\n# AMS HBase pid
> directory\nexport AMS_HBASE_PID_DIR\u003d{{hbase_pid_dir}}\n\n# AMS Collector
> heapsize\nexport A
MS_COLLECTOR_HEAPSIZE\u003d{{metrics_collector_heapsize}}\n\n# HBase
normalizer enabled\nexport
AMS_HBASE_NORMALIZER_ENABLED\u003d{{ams_hbase_normalizer_enabled}}\n\n# HBase
compaction policy enabled\nexport
AMS_HBASE_FIFO_COMPACTION_ENABLED\u003d{{ams_hbase_fifo_compaction_enabled}}\n\n#
HBase Tables Initialization check enabled\nexport
AMS_HBASE_INIT_CHECK_ENABLED\u003d{{ams_hbase_init_check_enabled}}\n\n# AMS
Collector options\nexport
AMS_COLLECTOR_OPTS\u003d\"-Djava.library.path\u003d/usr/lib/ams-hbase/lib/hadoop-native\"\n{%
if security_enabled %}\nexport AMS_COLLECTOR_OPTS\u003d\"$AMS_COLLECTOR_OPTS
-Djava.security.auth.login.config\u003d{{ams_collector_jaas_config_file}}\"\n{%
endif %}\n\n# AMS Collector GC options\nexport
AMS_COLLECTOR_GC_OPTS\u003d\"-XX:+UseConcMarkSweepGC -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCDateStamps
-Xloggc:{{ams_collector_log_dir}}/collector-gc.log-`date
+\u0027%Y%m%d%H%M\u0027`\"\nexport
AMS_COLLECTOR_OPTS\u003d\"$AMS_COLLECTOR_OPTS $AMS_COLLEC
TOR_GC_OPTS\"\n\n"}}
> ```
>
> Request response :
> ```
> {
> "status": 403,
> "message": "You do not have permissions to access this resource."
> }
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java
> eeb1a8b
>
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java
> ff47ac2
>
> Diff: https://reviews.apache.org/r/48181/diff/
>
>
> Testing
> ---
>
> Manaually tested
>
> # Local test results: PASSED
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Review Request 48204: HDP-UTILs Repo URL validation/save fails
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48204/ --- Review request for Ambari, Jonathan Hurley and Robert Levas. Bugs: AMBARI-17028 https://issues.apache.org/jira/browse/AMBARI-17028 Repository: ambari Description --- HDP-UTILS for HDP-2.5 was moved to 1.1.0.21 without our knowledge, so the API only recognized updating 1.1.0.20. Diffs - ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml 2119413 Diff: https://reviews.apache.org/r/48204/diff/ Testing --- No automated testing, it's just XML change. Manual tested following: - Default Install of 2.5.0.0 (make no URL changes, just "click through"). - Install 2.5.0.0, but change the URL to the version previous Thanks, Nate Cole
Re: Review Request 48162: 16171 Addendum2 for stackadvisor with Phoenix Query Server kerberos configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48162/#review136057 --- Ship it! ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 209) <https://reviews.apache.org/r/48162/#comment201043> %s % syntax is not the standard anymore, use "".format(...) - Nate Cole On June 2, 2016, 12:38 p.m., Josh Elser wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48162/ > --- > > (Updated June 2, 2016, 12:38 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas. > > > Bugs: AMBARI-16171 > https://issues.apache.org/jira/browse/AMBARI-16171 > > > Repository: ambari > > > Description > --- > > Fix an issue where unsubstituted variables are being left in core-site.xml > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json > f887f92 > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json > 5cfe42d > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 413a2f7 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > d61de7b > > Diff: https://reviews.apache.org/r/48162/diff/ > > > Testing > --- > > Deployed 2.4 with Kerberos and no PQS > Deployed 2.4 with Kerberos and PQS > (correct configs for both) > Python UTs are passing locally > java UTs still running locally > > > Thanks, > > Josh Elser > >
Re: Review Request 47783: Cluster operator and cluster admin not allowed to install ambari agent
> On May 24, 2016, 4:41 p.m., Nate Cole wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java, > > lines 190-194 > > <https://reviews.apache.org/r/47783/diff/1/?file=1392703#file1392703line190> > > > > Should special permissions like this go right in the action definition > > itself? Would require finding out if the file is readable by non-root > > Ambari. Would help with having to hard code action names here. > > Robert Levas wrote: > I dont think I understand the issue. > > The request to create a Request resource with the command "check_host" > needs to be processed to ensure that the user requesting this operation is > authorized to do so. This check cannot be done anywhere else since we dont > know until this point what the user is trying to do - that is without parsing > the request data an additional time just for the authorization check. I really only meant that check_host, and all other custom actions, are defined in a-s/src/main/resources/custom_action_definitions. Should the permissions needed to run be set with the definition, not special-cased in code? - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47783/#review134630 --- On May 24, 2016, 1:48 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47783/ > --- > > (Updated May 24, 2016, 1:48 p.m.) > > > Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Nate > Cole. > > > Bugs: AMBARI-16851 > https://issues.apache.org/jira/browse/AMBARI-16851 > > > Repository: ambari > > > Description > --- > > Cluster operator and the cluster admin must be allowed to add/delete hosts > but install of agents using /bootstrap fails with 403 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java > 5b318af > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java > 5c74f07 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > f4f614e > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 2c2d743 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql ee87cc5 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql a65df9c > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 6f38ec8 > ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql > ca57de5 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql bd2e6d6 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 9269b13 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java > 65efc63 > > ambari-server/src/test/java/org/apache/ambari/server/security/TestAuthenticationFactory.java > 69b4b08 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > 6511cb4 > > Diff: https://reviews.apache.org/r/47783/diff/ > > > Testing > --- > > Manually tested, newly created cluster and upgrade > > # Local test results: > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 1:15:22.164s > [INFO] Finished at: Tue May 24 13:28:21 EDT 2016 > [INFO] Final Memory: 59M/1807M > [INFO] > > > #Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 47785: Ambari install of Atlas should use external HBase and Logsearch SOLR
> On May 25, 2016, 7:30 p.m., Srimanth Gunturi wrote: > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py, > > line 150 > > <https://reviews.apache.org/r/47785/diff/2/?file=1394010#file1394010line150> > > > > Just wanted to make sure if this always be true? I think this assumption should be ok - we are already doing the symlink magic to point to the right spot. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47785/#review134876 --- On May 25, 2016, 6:06 p.m., Tom Beerbower wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47785/ > --- > > (Updated May 25, 2016, 6:06 p.m.) > > > Review request for Ambari, John Speidel, Nate Cole, and Srimanth Gunturi. > > > Bugs: AMBARI-16853 and ATLAS-823 > https://issues.apache.org/jira/browse/AMBARI-16853 > https://issues.apache.org/jira/browse/ATLAS-823 > > > Repository: ambari > > > Description > --- > > To support this, add dependency from Atlas to LogSearch SOLR. Also remove > references to embedded SOLR and embedded HBase. > > Use solr_cloud_util to create indexes for Atlas install. > > See AMBARI-15865. > See Atlas-823. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > bf0467e > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml > dd4b3e2 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-hbase-site.xml > 3c4826d > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/metainfo.xml > f4115f7 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py > 6b045ca > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py > e305138 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py > f172b79 > > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml > 0631b7d > > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/atlas-env.xml > 2a5f777 > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml > 15daeea > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > af812fe > ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py > 98fc678 > ambari-server/src/test/python/stacks/2.3/configs/default.json c8b418b > ambari-server/src/test/python/stacks/2.3/configs/secure.json 7ebcedf > ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 9d0f00c > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > 51c0d0c > ambari-server/src/test/python/stacks/2.5/configs/default.json e4a97f6 > > Diff: https://reviews.apache.org/r/47785/diff/ > > > Testing > --- > > Manual test of Atlas install. > > Updated unit tests. > > mvn clean test > > > Thanks, > > Tom Beerbower > >
Re: Review Request 48516: SPNEGO keytab and principal configuration for HBase web UIs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48516/#review136989 --- Ship it! Ship It! - Nate Cole On June 9, 2016, 7 p.m., Josh Elser wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48516/ > --- > > (Updated June 9, 2016, 7 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas. > > > Bugs: AMBARI-17129 > https://issues.apache.org/jira/browse/AMBARI-17129 > > > Repository: ambari > > > Description > --- > > Adds the SPNEGO principal and keytab to hbase-site.xml to make it simple for > users to enable SPNEGO authentication for HBase web UIs > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 5016325 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json > 50a7ceb > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json > 8be8bda > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > c221138 > > Diff: https://reviews.apache.org/r/48516/diff/ > > > Testing > --- > > Java unit tests and a simple virtual-machine installation of 2.4.0. Verified > that expected configuration properties are present. > > > Thanks, > > Josh Elser > >
Review Request 48657: Allow option to skip duplicate URL checking when creating VDF (part 2)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48657/ --- Review request for Ambari and Jonathan Hurley. Bugs: AMBARI-17173 https://issues.apache.org/jira/browse/AMBARI-17173 Repository: ambari Description --- My original patch had the for loop inside the check block, preventing an addition to the critical Set. Sigh. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java 62568cf ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java 3bc4aec Diff: https://reviews.apache.org/r/48657/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
Re: Review Request 48516: SPNEGO keytab and principal configuration for HBase web UIs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48516/#review137497 --- What is the status of this review? - Nate Cole On June 9, 2016, 7 p.m., Josh Elser wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48516/ > --- > > (Updated June 9, 2016, 7 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas. > > > Bugs: AMBARI-17129 > https://issues.apache.org/jira/browse/AMBARI-17129 > > > Repository: ambari > > > Description > --- > > Adds the SPNEGO principal and keytab to hbase-site.xml to make it simple for > users to enable SPNEGO authentication for HBase web UIs > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 5016325 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json > 50a7ceb > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json > 8be8bda > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > c221138 > > Diff: https://reviews.apache.org/r/48516/diff/ > > > Testing > --- > > Java unit tests and a simple virtual-machine installation of 2.4.0. Verified > that expected configuration properties are present. > > > Thanks, > > Josh Elser > >
Re: Review Request 48702: Add ability to set GET request directives
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48702/#review137568 --- Can you give an example of a read directive? ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java (line 333) <https://reviews.apache.org/r/48702/#comment202728> getReadDirectives() to match verb-to-action (we don't have getPostDirectives(), we have getCreateDirectives() ) ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java (line 69) <https://reviews.apache.org/r/48702/#comment202729> createReadRequest() ? - Nate Cole On June 14, 2016, 2:57 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48702/ > --- > > (Updated June 14, 2016, 2:57 p.m.) > > > Review request for Ambari, Ajit Kumar, Di Li, Jonathan Hurley, and Nate Cole. > > > Bugs: AMBARI-17224 > https://issues.apache.org/jira/browse/AMBARI-17224 > > > Repository: ambari > > > Description > --- > > Add ability to set GET request directives so that directives may be specified > in a GET requests. For example, to limit or increase the amount of work > needed to be performed internally to generate the data being returned. > > Most data in GET requests are retrieved but filtered from the response based > on the predicate. This means that any data the requires significant > processing to calculate will be calculated regardless of the requested > fields. By using a directive this can be controlled. > > Note: It may be possible the use of a PropertyProvider to control this > instead, but that may not be desired in all cases. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/handlers/ReadHandler.java > 95e45d6 > ambari-server/src/main/java/org/apache/ambari/server/api/query/Query.java > fff842a > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > 18fd94b > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java > dcc5217 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceDefinition.java > 8df1a02 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/SimpleResourceDefinition.java > 394e295 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/UpgradeResourceDefinition.java > bf7860b > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java > 0ebeb19 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java > 19fee8e > > ambari-server/src/test/java/org/apache/ambari/server/api/handlers/ReadHandlerTest.java > 5cb601e > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java > f174cb6 > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/SimpleResourceDefinitionTest.java > 6169ee7 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/RequestFactoryTest.java > 11568c9 > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/AuditEventCreatorTestHelper.java > 2642418 > > Diff: https://reviews.apache.org/r/48702/diff/ > > > Testing > --- > > # Local test results: > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 1:25:36.424s > [INFO] Finished at: Tue Jun 14 14:53:08 EDT 2016 > [INFO] Final Memory: 59M/1864M > [INFO] > > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 48702: Add ability to set GET request directives
> On June 14, 2016, 3:24 p.m., Nate Cole wrote: > > ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java, > > line 69 > > <https://reviews.apache.org/r/48702/diff/1/?file=1418983#file1418983line69> > > > > createReadRequest() ? > > Robert Levas wrote: > This is consistent with `createPutRequest`, `createDeleteRequest`, and > `createPostRequest`. Whoops - I read a little fast I guess :) - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48702/#review137568 --- On June 14, 2016, 2:57 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48702/ > --- > > (Updated June 14, 2016, 2:57 p.m.) > > > Review request for Ambari, Ajit Kumar, Di Li, Jonathan Hurley, and Nate Cole. > > > Bugs: AMBARI-17224 > https://issues.apache.org/jira/browse/AMBARI-17224 > > > Repository: ambari > > > Description > --- > > Add ability to set GET request directives so that directives may be specified > in a GET requests. For example, to limit or increase the amount of work > needed to be performed internally to generate the data being returned. > > Most data in GET requests are retrieved but filtered from the response based > on the predicate. This means that any data the requires significant > processing to calculate will be calculated regardless of the requested > fields. By using a directive this can be controlled. > > Note: It may be possible the use of a PropertyProvider to control this > instead, but that may not be desired in all cases. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/handlers/ReadHandler.java > 95e45d6 > ambari-server/src/main/java/org/apache/ambari/server/api/query/Query.java > fff842a > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > 18fd94b > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java > dcc5217 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceDefinition.java > 8df1a02 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/SimpleResourceDefinition.java > 394e295 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/UpgradeResourceDefinition.java > bf7860b > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java > 0ebeb19 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java > 19fee8e > > ambari-server/src/test/java/org/apache/ambari/server/api/handlers/ReadHandlerTest.java > 5cb601e > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java > f174cb6 > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/SimpleResourceDefinitionTest.java > 6169ee7 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/RequestFactoryTest.java > 11568c9 > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/AuditEventCreatorTestHelper.java > 2642418 > > Diff: https://reviews.apache.org/r/48702/diff/ > > > Testing > --- > > # Local test results: > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 1:25:36.424s > [INFO] Finished at: Tue Jun 14 14:53:08 EDT 2016 > [INFO] Final Memory: 59M/1864M > [INFO] > > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 48702: Add ability to set GET request directives
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48702/#review137720 --- Ship it! Ship It! - Nate Cole On June 14, 2016, 7:12 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48702/ > --- > > (Updated June 14, 2016, 7:12 p.m.) > > > Review request for Ambari, Ajit Kumar, Di Li, Jonathan Hurley, and Nate Cole. > > > Bugs: AMBARI-17224 > https://issues.apache.org/jira/browse/AMBARI-17224 > > > Repository: ambari > > > Description > --- > > Add ability to set GET request directives so that directives may be specified > in a GET requests. For example, to limit or increase the amount of work > needed to be performed internally to generate the data being returned. > > Most data in GET requests are retrieved but filtered from the response based > on the predicate. This means that any data the requires significant > processing to calculate will be calculated regardless of the requested > fields. By using a directive this can be controlled. > > Note: It may be possible the use of a PropertyProvider to control this > instead, but that may not be desired in all cases. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/handlers/ReadHandler.java > 95e45d6 > ambari-server/src/main/java/org/apache/ambari/server/api/query/Query.java > fff842a > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > 18fd94b > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java > dcc5217 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceDefinition.java > 8df1a02 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/SimpleResourceDefinition.java > 394e295 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/UpgradeResourceDefinition.java > bf7860b > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java > 0ebeb19 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java > 19fee8e > > ambari-server/src/test/java/org/apache/ambari/server/api/handlers/ReadHandlerTest.java > 5cb601e > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java > f174cb6 > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/SimpleResourceDefinitionTest.java > 6169ee7 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/RequestFactoryTest.java > 11568c9 > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/AuditEventCreatorTestHelper.java > 2642418 > > Diff: https://reviews.apache.org/r/48702/diff/ > > > Testing > --- > > # Local test results: > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 1:25:36.424s > [INFO] Finished at: Tue Jun 14 14:53:08 EDT 2016 > [INFO] Final Memory: 59M/1864M > [INFO] > > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 48557: Fixed implementation of on-ambari-upgrade support. Patch 2: add logic for ambari-upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48557/#review137357 --- Ship it! Ship It! - Nate Cole On June 13, 2016, 9:22 a.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48557/ > --- > > (Updated June 13, 2016, 9:22 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, > and Sumit Mohanty. > > > Bugs: AMBARI-17112 > https://issues.apache.org/jira/browse/AMBARI-17112 > > > Repository: ambari > > > Description > --- > > Patch implements logic for on-ambari-upgrade attributes: add, update, delete > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java > de5147d > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > c570ab3 > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java > f6791ee > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 3ee8bba > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-site.xml > 6793e4e > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > fd68ae0 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-startup.properties.xml > aacf10a > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-site.xml > 4e56084 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/core-site.xml > 7f67b8a > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml > 274163e > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml > b0f36e7 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml > 444b8b4 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml > cfdd989 > > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml > 7f474f6 > > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml > f3a6bcf > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml > 47c900e > > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml > a309fa4 > > ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-site.xml > 2d0047c > > ambari-server/src/main/resources/common-services/RANGER/0.4.0/configuration/ranger-env.xml > 5082277 > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml > 121a797 > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml > ae56f8b > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml > 4317cfa > > ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml > 7eb78e5 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml > a747dde > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 35910ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml > 6eb312f > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 9870b8b > > ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml > 633cbda > > ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml > daacdee > > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml > 21cd096 > > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml > a8f1584 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml > d7c87e9 >
Re: Review Request 48655: ATLAS conf dir needs to be present in all ATLAS hook deployed hosts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48655/#review137359 --- Ship it! Ship It! - Nate Cole On June 13, 2016, 1:51 p.m., Tom Beerbower wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48655/ > --- > > (Updated June 13, 2016, 1:51 p.m.) > > > Review request for Ambari, John Speidel and Nate Cole. > > > Bugs: AMBARI-17203 > https://issues.apache.org/jira/browse/AMBARI-17203 > > > Repository: ambari > > > Description > --- > > Atlas atlas-application.properties configuration is required by the Atlas > hooks (Falcon, Storm, Sqoop and Hive). > Currently the atlas configuration is not being pushed to all the hook hosts > which could lead to issues if any config is changed. The Atlas configuration > changes needs to be pushed to all dependent hosts. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py > 441f0da > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py > 67077c4 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml > c20523d > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py > fea0635 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py > e78190f > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py > c154a16 > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py > d18d820 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py > eadbd4a > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py > 6c3e91f > > ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/metainfo.xml > 1998131 > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml > ac1f9e7 > ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/metainfo.xml > eb67d63 > ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/metainfo.xml > 3faadc0 > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 1bcc09e > > Diff: https://reviews.apache.org/r/48655/diff/ > > > Testing > --- > > Manual test deploy Atlas with all hook integration services on multi-host > cluster. Verify Atlas configuration locations. Update Atlas configuration. > Verify dependant service restart suggestion from Ambari. > > mvn clean test > > > Thanks, > > Tom Beerbower > >
Re: Review Request 48640: Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to SERVICE.ADMINISTRATOR role and above
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48640/#review137358
---
Ship it!
Ship It!
- Nate Cole
On June 13, 2016, 2:50 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48640/
> ---
>
> (Updated June 13, 2016, 2:50 p.m.)
>
>
> Review request for Ambari, Eugene Chekanskiy, Jonathan Hurley, Nate Cole, and
> Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-17177
> https://issues.apache.org/jira/browse/AMBARI-17177
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Add SERVICE.VIEW_OPERATIONAL_LOGS authorization to the following roles:
>
> * AMBARI.ADMINISTRATOR
> * CLUSTER.ADMINISTRATOR
> * CLUSTER.OPERATOR
> * SERVICE.ADMINISTRATOR
>
> This is a DB change adding an authorization record to the `roleauthorization`
> table and relevant records for the different roles to the
> `permission_roleauthorization` table.
>
> The description/name of the `SERVICE.VIEW_OPERATIONAL_LOGS` authorization
> should be
> ```
> View service operational logs
> ```
>
>
> Diffs
> -
>
>
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js
> 33ed7ed
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RoleAuthorizationDAO.java
> aa74224
>
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java
> ee948fe
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
> 3ee8bba
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog230.java
> 6b038f4
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
> 5016325
> ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql fff1716
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql e7eff93
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b90c7da
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql b9181d2
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql
> c840b9c
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 742eaa3
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 87ef7fb
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog230Test.java
> d7e13ea
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
> c221138
>
> Diff: https://reviews.apache.org/r/48640/diff/
>
>
> Testing
> ---
>
> Manually tested upgrade and new cluster creation
>
> # Local test results:
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 1:16:17.022s
> [INFO] Finished at: Mon Jun 13 09:41:19 EDT 2016
> [INFO] Final Memory: 59M/1881M
> [INFO]
>
>
> # Jenkins test results:
>
> Tests run: 4478, Failures: 0, Errors: 0, Skipped: 34
>
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 01:38 h
> [INFO] Finished at: 2016-06-13T17:03:59+00:00
> [INFO] Final Memory: 152M/527M
> [INFO]
>
>
>
> {color:green}+1 overall{color}. Here are the results of testing the latest
> attachment
>
> http://issues.apache.org/jira/secure/attachment/12809853/AMBARI-17177_trunk_01.patch
> against trunk revision .
>
> {color:green}+1 @author{color}. The patch does not contain any @author
> tags.
>
> {color:green}+1 tests included{color}. The patch appears to include 2
> new or modified test files.
>
> {color:green}+1 javac{color}. The applied patch does not increase the
> total number of javac compiler warnings.
>
> {color:green}+1 release audit{color}. The applied patch does not
> increase the total number of release audit warnings.
>
> {color:green}+1 core tests{color}. The patch passed unit tests in
> ambari-admin ambari-server.
>
> Test results:
> https://builds.apache.org/job/Ambari-trunk-test-patch/7322//testReport/
> Console output:
> https://builds.apache.org/job/Ambari-trunk-test-patch/7322//console
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 48415: Authorizations given to role-based principals must be dereferenced upon user login
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48415/#review136644 --- Ship it! Ship It! - Nate Cole On June 8, 2016, 9:53 a.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48415/ > --- > > (Updated June 8, 2016, 9:53 a.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jonathan Hurley, Myroslav > Papirkovskyy, and Nate Cole. > > > Bugs: AMBARI-16247 > https://issues.apache.org/jira/browse/AMBARI-16247 > > > Repository: ambari > > > Description > --- > > Authorizations given to role-based principals must be dereferenced upon user > login. These authorizations are dynamically determined based on the user's > set of roles. > > In > `org.apache.ambari.server.security.authorization.AmbariLocalUserDetailsService#loadUserByUsername`, > the set of `GrantedAuthorities` the authenticated user is calculated. > During this process, using the set of `cluster-level roles` a user is > granted, any permissions given to matching role-based principals should be > given to the user. > > This essentially work like giving privileges to a group of users calculated > at runtime. > > A use-case to support the need for this is to assign access to a view to all > users with some specific role. Currently we can assign access to a view to a > specific user or group by assigning that user or group the `VIEW.USER` role > applied to the specific view. To assign access a view to users who have a > specific role, a `role` will need to behave like a `principal`. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java > 545095d > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/UsersTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/48415/diff/ > > > Testing > --- > > Manually tested > > # Local test results: PENDING > > # Jenkinks test results: PENDING > > > Thanks, > > Robert Levas > >
Review Request 48088: Stack VDF should ignore unsupported OS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48088/ --- Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Jayush Luniya. Bugs: AMBARI-16971 https://issues.apache.org/jira/browse/AMBARI-16971 Repository: ambari Description --- Stack VDF loaded via hdp_urlinfo.json may contain OS families that are not supported by the stack. They should not be included in the merged Version Definition Diffs - ambari-common/src/main/python/ambari_commons/resources/os_family.json c7b38aa ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java a332308 ambari-server/src/main/java/org/apache/ambari/server/state/stack/OsFamily.java e494c44 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java 9f7a90f ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json f80fc0a ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.4-124-suse11.xml PRE-CREATION Diff: https://reviews.apache.org/r/48088/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
Re: Review Request 47933: Add conditional constraints for Kerberos identities to control when they are created
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47933/#review135255
---
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/AndPredicate.java
(lines 19 - 24)
<https://reviews.apache.org/r/47933/#comment200277>
Seems like these are more generic than just for kerberos - maybe should go
in org.apache.ambari.collections or something? Is there no other guice/guava
library doing the same stuff (json query framework or something)?
- Nate Cole
On May 27, 2016, 6:01 a.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47933/
> ---
>
> (Updated May 27, 2016, 6:01 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sandor
> Magyari, Sumit Mohanty, and Swapan Shridhar.
>
>
> Bugs: AMBARI-16437
> https://issues.apache.org/jira/browse/AMBARI-16437
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Add conditional constraints for Kerberos identities to control when they are
> created. For example if Kerberos Identity should only be created (and
> distributed) for a component when some other component or service is
> installed.
>
> An example of this would be
> ```
> {
> "name": "/HIVE/HIVE_SERVER/hive_server_hive",
> "principal": {
> "configuration":
> "hive-interactive-site/hive.llap.daemon.service.principal"
> },
> "keytab": {
> "configuration": "hive-interactive-site/hive.llap.daemon.keytab.file"
> },
> "when" : {
> "contains" : ["services", "HIVE"]
> }
> }
> ```
>
> Note the "`when`" clause. This indicates that this identity should only be
> processed when the set of services contains "HIVE". An alternative to this
> would be to test the set of components for a certain component.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
> c67c55d
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
> 0dbd357
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
> bb2ed1c
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
> d31dd21
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/AndPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/ContainsPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/ContextTransformer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/DelegatedMultiplePredicateContainer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/DelegatedSinglePredicateContainer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/EqualsPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/NotPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/OperationPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/OrPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/Predicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/PredicateClassFactory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/PredicateUtils.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
> 5393fd6
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
> d80d7cc
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
> 0ea7b26
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/eval/AndPredicateTest.java
> PRE-CREATION
>
Re: Review Request 47933: Add conditional constraints for Kerberos identities to control when they are created
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47933/#review135256
---
Ship it!
Ship It!
- Nate Cole
On May 27, 2016, 6:01 a.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47933/
> ---
>
> (Updated May 27, 2016, 6:01 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sandor
> Magyari, Sumit Mohanty, and Swapan Shridhar.
>
>
> Bugs: AMBARI-16437
> https://issues.apache.org/jira/browse/AMBARI-16437
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Add conditional constraints for Kerberos identities to control when they are
> created. For example if Kerberos Identity should only be created (and
> distributed) for a component when some other component or service is
> installed.
>
> An example of this would be
> ```
> {
> "name": "/HIVE/HIVE_SERVER/hive_server_hive",
> "principal": {
> "configuration":
> "hive-interactive-site/hive.llap.daemon.service.principal"
> },
> "keytab": {
> "configuration": "hive-interactive-site/hive.llap.daemon.keytab.file"
> },
> "when" : {
> "contains" : ["services", "HIVE"]
> }
> }
> ```
>
> Note the "`when`" clause. This indicates that this identity should only be
> processed when the set of services contains "HIVE". An alternative to this
> would be to test the set of components for a certain component.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
> c67c55d
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
> 0dbd357
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
> bb2ed1c
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
> d31dd21
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/AndPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/ContainsPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/ContextTransformer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/DelegatedMultiplePredicateContainer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/DelegatedSinglePredicateContainer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/EqualsPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/NotPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/OperationPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/OrPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/Predicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/PredicateClassFactory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/eval/PredicateUtils.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
> 5393fd6
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
> d80d7cc
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
> 0ea7b26
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/eval/AndPredicateTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/eval/ContainsPredicateTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/eval/ContextTransformerTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/eval/EqualsPredicateTest.java
>
Re: Review Request 47961: Web Client Requests Handled By Jetty Should Not Be Blocked By JMX Property Providers
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47961/#review135261
---
Ship it!
Ship It!
- Nate Cole
On May 27, 2016, 12:26 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47961/
> ---
>
> (Updated May 27, 2016, 12:26 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas,
> Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-16913
> https://issues.apache.org/jira/browse/AMBARI-16913
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Incoming requests from the web client (or from any REST API) will eventually
> be routed to the property provider / subresource framework. It is here were
> any JMX data is queried for within the context of the REST request. In large
> clusters, these requests can backup quite easily (even with a massive
> threadpool), causing UX degradations in the web client:
>
> ```
> Thread [qtp-ambari-client-38]
>
> JMXPropertyProvider(ThreadPoolEnabledPropertyProvider).populateResources(Set,
> Request, Predicate) line: 168
> JMXPropertyProvider.populateResources(Set, Request,
> Predicate) line: 156
> StackDefinedPropertyProvider.populateResources(Set, Request,
> Predicate) line: 200
> ClusterControllerImpl.populateResources(Type, Set, Request,
> Predicate) line: 155
> QueryImpl.queryForResources() line: 407
> QueryImpl.execute() line: 217
> ReadHandler.handleRequest(Request) line: 69
> GetRequest(BaseRequest).process() line: 145
> ```
>
> Consider one of the calls made by the web client:
> ```
> GET api/v1/clusters/c1/components/?
> ServiceComponentInfo/category=MASTER&
> fields=
> ServiceComponentInfo/service_name,
> host_components/HostRoles/display_name,
> host_components/HostRoles/host_name,
> host_components/HostRoles/state,
> host_components/HostRoles/maintenance_state,
> host_components/HostRoles/stale_configs,
> host_components/HostRoles/ha_state,
> host_components/HostRoles/desired_admin_state,
> host_components/metrics/jvm/memHeapUsedM,
> host_components/metrics/jvm/HeapMemoryMax,
> host_components/metrics/jvm/HeapMemoryUsed,
> host_components/metrics/jvm/memHeapCommittedM,
> host_components/metrics/mapred/jobtracker/trackers_decommissioned,
> host_components/metrics/cpu/cpu_wio,
> host_components/metrics/rpc/client/RpcQueueTime_avg_time,
> host_components/metrics/dfs/FSNamesystem/*,
> host_components/metrics/dfs/namenode/Version,
> host_components/metrics/dfs/namenode/LiveNodes,
> host_components/metrics/dfs/namenode/DeadNodes,
> host_components/metrics/dfs/namenode/DecomNodes,
> host_components/metrics/dfs/namenode/TotalFiles,
> host_components/metrics/dfs/namenode/UpgradeFinalized,
> host_components/metrics/dfs/namenode/Safemode,
> host_components/metrics/runtime/StartTime
> ```
>
> This query is essentially saying that for every {{MASTER}}, get metrics from
> them. The problem is that in a large cluster, there could be 100 masters, yet
> the metrics being asked for are only for NameNode. As a result, the JMX
> endpoints for all 100 masters are queried - *live* - as part of the request.
>
> There are two inherent flaws with this approach:
>
> - Even with millisecond JMX response times, multiplying this by 100's and
> then adding parsing overhead causes a noticeable delay in the web client as
> the federated requests are blocking the main UX request
>
> - Although there is a threadpool which scales up to service these requests -
> that only really works for 1 user. With multiple users logged in, you'd need
> 100's upon 100's of threads pulling in the same JMX data
>
> This data should never be queried for directly as part of the incoming REST
> requests. Instead, an autonomous pool of threads should be constantly
> retrieving these point-in-time metrics and updating a cache. The cache is
> then used to service all live REST requests.
> - On the first request to a resource, a cache miss occurs and no data is
> returned. I think this is acceptable since metrics take a few moments to
> populate anyway right now. As the web client polls, the next request should
> pickup the newly cached metrics.
> - Only URLs which are being asked for by incoming REST requests should be
> considered for retrieval. After sometime, if they haven't been requested,
>
Re: Review Request 47961: Web Client Requests Handled By Jetty Should Not Be Blocked By JMX Property Providers
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47961/#review135268
---
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
(lines 138 - 145)
<https://reviews.apache.org/r/47961/#comment200286>
Doesn't seem like you need 2 of these that do the same thing. REST vs JMX
doesn't appear to matter.
ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
(lines 395 - 401)
<https://reviews.apache.org/r/47961/#comment200287>
nit: these don't throw anything
ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
(lines 430 - 432)
<https://reviews.apache.org/r/47961/#comment200288>
nit: you have two Runnables that do nearly the identical thing except how
to parse the resulting InputStream. Could push most of the run() to
MetricsRunnable and just have your subclasses parse.
- Nate Cole
On May 27, 2016, 1:27 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47961/
> ---
>
> (Updated May 27, 2016, 1:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, Robert Levas,
> Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-16913
> https://issues.apache.org/jira/browse/AMBARI-16913
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Incoming requests from the web client (or from any REST API) will eventually
> be routed to the property provider / subresource framework. It is here were
> any JMX data is queried for within the context of the REST request. In large
> clusters, these requests can backup quite easily (even with a massive
> threadpool), causing UX degradations in the web client:
>
> ```
> Thread [qtp-ambari-client-38]
>
> JMXPropertyProvider(ThreadPoolEnabledPropertyProvider).populateResources(Set,
> Request, Predicate) line: 168
> JMXPropertyProvider.populateResources(Set, Request,
> Predicate) line: 156
> StackDefinedPropertyProvider.populateResources(Set, Request,
> Predicate) line: 200
> ClusterControllerImpl.populateResources(Type, Set, Request,
> Predicate) line: 155
> QueryImpl.queryForResources() line: 407
> QueryImpl.execute() line: 217
> ReadHandler.handleRequest(Request) line: 69
> GetRequest(BaseRequest).process() line: 145
> ```
>
> Consider one of the calls made by the web client:
> ```
> GET api/v1/clusters/c1/components/?
> ServiceComponentInfo/category=MASTER&
> fields=
> ServiceComponentInfo/service_name,
> host_components/HostRoles/display_name,
> host_components/HostRoles/host_name,
> host_components/HostRoles/state,
> host_components/HostRoles/maintenance_state,
> host_components/HostRoles/stale_configs,
> host_components/HostRoles/ha_state,
> host_components/HostRoles/desired_admin_state,
> host_components/metrics/jvm/memHeapUsedM,
> host_components/metrics/jvm/HeapMemoryMax,
> host_components/metrics/jvm/HeapMemoryUsed,
> host_components/metrics/jvm/memHeapCommittedM,
> host_components/metrics/mapred/jobtracker/trackers_decommissioned,
> host_components/metrics/cpu/cpu_wio,
> host_components/metrics/rpc/client/RpcQueueTime_avg_time,
> host_components/metrics/dfs/FSNamesystem/*,
> host_components/metrics/dfs/namenode/Version,
> host_components/metrics/dfs/namenode/LiveNodes,
> host_components/metrics/dfs/namenode/DeadNodes,
> host_components/metrics/dfs/namenode/DecomNodes,
> host_components/metrics/dfs/namenode/TotalFiles,
> host_components/metrics/dfs/namenode/UpgradeFinalized,
> host_components/metrics/dfs/namenode/Safemode,
> host_components/metrics/runtime/StartTime
> ```
>
> This query is essentially saying that for every {{MASTER}}, get metrics from
> them. The problem is that in a large cluster, there could be 100 masters, yet
> the metrics being asked for are only for NameNode. As a result, the JMX
> endpoints for all 100 masters are queried - *live* - as part of the request.
>
> There are two inherent flaws with this approach:
>
> - Even with millisecond JMX response times, multiplying this by 100's and
> then adding parsing overhead causes a noticeable delay in the web client as
> the federated requests are blocking the main UX request
>
> - Although there is a threadpool which scales up to service these requests -
> that only re
Re: Review Request 47933: Add conditional constraints for Kerberos identities to control when they are created
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47933/#review135300
---
Ship it!
Ship It!
- Nate Cole
On May 27, 2016, 3:58 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47933/
> ---
>
> (Updated May 27, 2016, 3:58 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sandor
> Magyari, Sumit Mohanty, and Swapan Shridhar.
>
>
> Bugs: AMBARI-16437
> https://issues.apache.org/jira/browse/AMBARI-16437
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Add conditional constraints for Kerberos identities to control when they are
> created. For example if Kerberos Identity should only be created (and
> distributed) for a component when some other component or service is
> installed.
>
> An example of this would be
> ```
> {
> "name": "/HIVE/HIVE_SERVER/hive_server_hive",
> "principal": {
> "configuration":
> "hive-interactive-site/hive.llap.daemon.service.principal"
> },
> "keytab": {
> "configuration": "hive-interactive-site/hive.llap.daemon.keytab.file"
> },
> "when" : {
> "contains" : ["services", "HIVE"]
> }
> }
> ```
>
> Note the "`when`" clause. This indicates that this identity should only be
> processed when the set of services contains "HIVE". An alternative to this
> would be to test the set of components for a certain component.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/Predicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/PredicateUtils.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/AndPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/ContainsPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/ContextTransformer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/DelegatedMultiplePredicateContainer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/DelegatedSinglePredicateContainer.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/EqualsPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/NotPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/OperationPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/OrPredicate.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/collections/functors/PredicateClassFactory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
> c67c55d
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
> 0dbd357
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
> bb2ed1c
>
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
> d31dd21
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/PredicateUtilsTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/functors/AndPredicateTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/functors/ContainsPredicateTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/functors/ContextTransformerTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/functors/EqualsPredicateTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/functors/NotPredicateTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/collections/functors/OrPredicateTes
Re: Review Request 47979: Deploy: UI: Hive_metastore not started
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47979/#review135319
---
Ship it!
Should add tests covering both hive/hive2.
- Nate Cole
On May 27, 2016, 4:15 p.m., Vitalyi Brodetskyi wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47979/
> ---
>
> (Updated May 27, 2016, 4:15 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Nate Cole, and Sumit
> Mohanty.
>
>
> Bugs: AMBARI-16938
> https://issues.apache.org/jira/browse/AMBARI-16938
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Deploy ambari cluster using UI with hive for existing mysql setup
> Actual: Hive metastore is failing to start due to unable to connect to DB.
> 2016-05-26 00:20:15,442 - Execute['/usr/jdk64/jdk1.8.0_77/bin/java -cp
> /usr/lib/ambari-agent/DBConnectionVerification.jar:/usr/hdp/current/hive-metastore/lib/mysql-connector-java.jar
> org.apache.ambari.server.DBConnectionVerification 'jdbc:mysql://host/hivedb'
> hiveuser PROTECTED com.mysql.jdbc.Driver']
> {'path': ['/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 5,
> 'try_sleep': 10}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
> abb8469
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_interactive.py
> 5639922
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py
> 7554010
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
> 76a417d
>
> Diff: https://reviews.apache.org/r/47979/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Review Request 47913: VDF builder script and XSD should be updated for package-version changes
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47913/
---
Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
Bugs: AMBARI-16909
https://issues.apache.org/jira/browse/AMBARI-16909
Repository: ambari
Description
---
The {{package-version}} element should belong with the {{os}} element such that
we can send that information to agents effectively when multiple OS are defined
in the XML. This will allow us to use specific build numbers when installing
packages without using wildcards.
This patch only covers builder + XSD. Ambari code to send this data to agents
is another JIRA.
Diffs
-
ambari-server/src/main/resources/version_definition.xsd 8c20c06
contrib/version-builder/example.py dd719bc
contrib/version-builder/example.sh ce60318
contrib/version-builder/version_builder.py 2e3fc7f
Diff: https://reviews.apache.org/r/47913/diff/
Testing
---
No new tests as majority of testing is in {{contrib}}. Automated testing
enough to verify existing XML still worked:
mvn test -Dtest=VersionDefinition*
---
T E S T S
---
Picked up JAVA_TOOL_OPTIONS: -Xmx2048m -XX:MaxPermSize=128m
Running
org.apache.ambari.server.controller.internal.VersionDefinitionResourceProviderTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.244 sec - in
org.apache.ambari.server.controller.internal.VersionDefinitionResourceProviderTest
Picked up JAVA_TOOL_OPTIONS: -Xmx2048m -XX:MaxPermSize=128m
Running org.apache.ambari.server.state.repository.VersionDefinitionTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.595 sec - in
org.apache.ambari.server.state.repository.VersionDefinitionTest
Results :
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0
Thanks,
Nate Cole
Re: Review Request 47867: RU: install version should be blocked while upgrade in progress
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47867/ --- (Updated May 26, 2016, 6:47 a.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-16889 https://issues.apache.org/jira/browse/AMBARI-16889 Repository: ambari Description (updated) --- Throw an error when POSTing to /api/v1/clusters//stack_versions when an upgrade/downgrade is in-progress Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java 05ee6c9 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 4a214e1 ambari-server/src/main/resources/version_definition.xsd b6360ee ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 8347a7b Diff: https://reviews.apache.org/r/47867/diff/ Testing (updated) --- Manual. Automated: Tests run: 4360, Failures: 0, Errors: 0, Skipped: 34 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 38:01.727s [INFO] Finished at: Wed May 25 21:30:01 EDT 2016 [INFO] Final Memory: 34M/709M [INFO] Thanks, Nate Cole
Re: Review Request 47867: RU: install version should be blocked while upgrade in progress
> On May 25, 2016, 10:02 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java, > > line 287 > > <https://reviews.apache.org/r/47867/diff/1/?file=1395048#file1395048line287> > > > > Nitpicking here, but should check size > 1 :) I had noticed this check was in an odd spot, so had only moved it. But you're right, a check of >1 is appropriate. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47867/#review134902 --- On May 25, 2016, 8:54 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47867/ > --- > > (Updated May 25, 2016, 8:54 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. > > > Bugs: AMBARI-16889 > https://issues.apache.org/jira/browse/AMBARI-16889 > > > Repository: ambari > > > Description > --- > > Throw an error when POSTing to /api/v1/clusters//stack_versions > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java > 05ee6c9 > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > 4a214e1 > ambari-server/src/main/resources/version_definition.xsd b6360ee > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 8347a7b > > Diff: https://reviews.apache.org/r/47867/diff/ > > > Testing > --- > > Manual. Automated pending > > > Thanks, > > Nate Cole > >
Re: Review Request 47871: Support Storm 1.0 in EU/RU to HDP 2.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47871/#review134974 --- Ship it! ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml (lines 897 - 899) <https://reviews.apache.org/r/47871/#comment199970> Why does downgrade get this message but not upgrade? - Nate Cole On May 25, 2016, 9:54 p.m., Alejandro Fernandez wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47871/ > --- > > (Updated May 25, 2016, 9:54 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan > Hurley, Nate Cole, and Swapan Shridhar. > > > Bugs: AMBARI-16648 > https://issues.apache.org/jira/browse/AMBARI-16648 > > > Repository: ambari > > > Description > --- > > HDP 2.5 is introducing breaking changes for Storm, so must apply config > changes and delete all local storm data plus data on Zookeeper. > EU and RU from HDP 2.3 -> 2.5 and 2.4 -> 2.5 must do the following > > Apply config changes > Delete Storm data on ZK only once > Delete Storm local data on all Storm hosts > > > Diffs > - > > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml > dd033b8 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml > 525a356 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml > 9ee3e88 > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml > 625b851 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml > 58aa95f > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml > cdb8634 > ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml > 5211276 > ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml > d90d76d > > Diff: https://reviews.apache.org/r/47871/diff/ > > > Testing > --- > > Testing on EU/RU from HDP 2.3 -> 2.5 and 2.4 -> 2.5 with Storm > > > Thanks, > > Alejandro Fernandez > >
Re: Review Request 47785: Ambari install of Atlas should use external HBase and Logsearch SOLR
> On May 25, 2016, 7:30 p.m., Srimanth Gunturi wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py,
> > line 150
> > <https://reviews.apache.org/r/47785/diff/2/?file=1394010#file1394010line150>
> >
> > Just wanted to make sure if this always be true?
>
> Nate Cole wrote:
> I think this assumption should be ok - we are already doing the symlink
> magic to point to the right spot.
>
> Tom Beerbower wrote:
> Thanks for the review Srimanth and Nate.
>
> Looking in the HBase status_params.py, I see this ...
>
> hbase_conf_dir = "/etc/hbase/conf"
> if stack_version_formatted and
> check_stack_feature(StackFeature.ROLLING_UPGRADE, stack_version_formatted):
> hbase_conf_dir =
> format("{stack_root}/current/{component_directory}/conf")
>
> Do, I need to add the same logic?
That's probably outside the scope of this fix, but that code you pointed out is
used such that when doing an upgrade, you are looking at the right place for
configs. I don't fully understand the relationship between Atlas and HBase (is
the dependence only on the client?), so that can probably be a different patch.
- Nate
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47785/#review134876
---
On May 25, 2016, 6:06 p.m., Tom Beerbower wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47785/
> -------
>
> (Updated May 25, 2016, 6:06 p.m.)
>
>
> Review request for Ambari, John Speidel, Nate Cole, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-16853 and ATLAS-823
> https://issues.apache.org/jira/browse/AMBARI-16853
> https://issues.apache.org/jira/browse/ATLAS-823
>
>
> Repository: ambari
>
>
> Description
> ---
>
> To support this, add dependency from Atlas to LogSearch SOLR. Also remove
> references to embedded SOLR and embedded HBase.
>
> Use solr_cloud_util to create indexes for Atlas install.
>
> See AMBARI-15865.
> See Atlas-823.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
> bf0467e
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
> dd4b3e2
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-hbase-site.xml
> 3c4826d
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/metainfo.xml
> f4115f7
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py
> 6b045ca
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
> e305138
>
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
> f172b79
>
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
> 0631b7d
>
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/atlas-env.xml
> 2a5f777
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml
> 15daeea
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
> af812fe
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py
> 98fc678
> ambari-server/src/test/python/stacks/2.3/configs/default.json c8b418b
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 7ebcedf
> ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 9d0f00c
> ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py
> 51c0d0c
> ambari-server/src/test/python/stacks/2.5/configs/default.json e4a97f6
>
> Diff: https://reviews.apache.org/r/47785/diff/
>
>
> Testing
> ---
>
> Manual test of Atlas install.
>
> Updated unit tests.
>
> mvn clean test
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 47785: Ambari install of Atlas should use external HBase and Logsearch SOLR
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47785/#review134975 --- Ship it! Ship It! - Nate Cole On May 25, 2016, 6:06 p.m., Tom Beerbower wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47785/ > --- > > (Updated May 25, 2016, 6:06 p.m.) > > > Review request for Ambari, John Speidel, Nate Cole, and Srimanth Gunturi. > > > Bugs: AMBARI-16853 and ATLAS-823 > https://issues.apache.org/jira/browse/AMBARI-16853 > https://issues.apache.org/jira/browse/ATLAS-823 > > > Repository: ambari > > > Description > --- > > To support this, add dependency from Atlas to LogSearch SOLR. Also remove > references to embedded SOLR and embedded HBase. > > Use solr_cloud_util to create indexes for Atlas install. > > See AMBARI-15865. > See Atlas-823. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > bf0467e > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml > dd4b3e2 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-hbase-site.xml > 3c4826d > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/metainfo.xml > f4115f7 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py > 6b045ca > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py > e305138 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py > f172b79 > > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml > 0631b7d > > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/atlas-env.xml > 2a5f777 > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml > 15daeea > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > af812fe > ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py > 98fc678 > ambari-server/src/test/python/stacks/2.3/configs/default.json c8b418b > ambari-server/src/test/python/stacks/2.3/configs/secure.json 7ebcedf > ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 9d0f00c > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > 51c0d0c > ambari-server/src/test/python/stacks/2.5/configs/default.json e4a97f6 > > Diff: https://reviews.apache.org/r/47785/diff/ > > > Testing > --- > > Manual test of Atlas install. > > Updated unit tests. > > mvn clean test > > > Thanks, > > Tom Beerbower > >
Re: Review Request 48036: Service name shown instead of Host name on popup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48036/#review135840 --- Ship it! Ship It! - Nate Cole On June 1, 2016, 12:01 p.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48036/ > --- > > (Updated June 1, 2016, 12:01 p.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-16953 > https://issues.apache.org/jira/browse/AMBARI-16953 > > > Repository: ambari > > > Description > --- > > STR: > # Deploy ambari with HDP2.2 > # Register and install HDP2.5 > # Click "Upgrade" > # Look at the Rolling Upgrade checks > Result: Service name SECONDARY_NAMENODE shown instead of Host name on popup > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java > 5899370 > > ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java > 4893098 > > ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java > 9abfa50 > > ambari-server/src/test/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheckTest.java > e2617bf > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServiceCheckValidityCheckTest.java > 55429bd > > Diff: https://reviews.apache.org/r/48036/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 47428: Changes to Phoenix QueryServer Kerberos configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47428/#review135792 --- Ship it! Ship It! - Nate Cole On May 27, 2016, 6:48 p.m., Josh Elser wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47428/ > --- > > (Updated May 27, 2016, 6:48 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and > Srimanth Gunturi. > > > Bugs: AMBARI-16171 > https://issues.apache.org/jira/browse/AMBARI-16171 > > > Repository: ambari > > > Description > --- > > The up-coming version of Phoenix will contain some new functionality to > support Kerberos authentication of clients via SPNEGO with the Phoenix Query > Server (PQS). > > Presently, Ambari will configure PQS to use the hbase service keytab which > will result in the SPNEGO authentication failing as the RFC requires that the > "primary" component of the Kerberos principal for the server is "HTTP". Thus, > we need to ensure that we switch PQS over to use the spnego.service.keytab as > the keytab and "HTTP/_HOST@REALM" as the principal. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 2e857ed > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 0deba5d > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json > c9536f8 > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 6e506a0 > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 8c5351f > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > 4dedc98 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > 0066e1d > > Diff: https://reviews.apache.org/r/47428/diff/ > > > Testing > --- > > Unit testing, verified installation with trunk and proper kerberization. > > > Thanks, > > Josh Elser > >
Re: Review Request 48036: Service name shown instead of Host name on popup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48036/#review135636 --- ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java (lines 102 - 103) <https://reviews.apache.org/r/48036/#comment200659> If this is getting changed to a HOST type, then the CheckDescription enum should be changed to PrereqCheckType.HOST. Hmm, but this is really a service-based check (SNN must be deleted). Maybe just make the host name where it was found to be informational rather than the getFailedOn(). And in fact, getFailedOn() would be HDFS, not SNN since it's a service and SNN is a component. Either way, should make sure a test covers this appropriately. - Nate Cole On May 30, 2016, 9:18 a.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48036/ > --- > > (Updated May 30, 2016, 9:18 a.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-16953 > https://issues.apache.org/jira/browse/AMBARI-16953 > > > Repository: ambari > > > Description > --- > > STR: > # Deploy ambari with HDP2.2 > # Register and install HDP2.5 > # Click "Upgrade" > # Look at the Rolling Upgrade checks > Result: Service name SECONDARY_NAMENODE shown instead of Host name on popup > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java > 4893098 > > Diff: https://reviews.apache.org/r/48036/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 48041: SERVICE_CHECK Upgrade pre-check does not throw error when its expected to
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48041/#review135637
---
ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java
(lines 152 - 155)
<https://reviews.apache.org/r/48041/#comment200660>
I think we should add an error message for the case where NO service check
was run, versus the message when no RECENT service check was run. See how
CLIENT_RETRY check works with a List of fail messages to see how to do it.
- Nate Cole
On May 30, 2016, 9:30 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48041/
> ---
>
> (Updated May 30, 2016, 9:30 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-16954
> https://issues.apache.org/jira/browse/AMBARI-16954
>
>
> Repository: ambari
>
>
> Description
> ---
>
> *Steps*
> # Deploy HDP-2.4.0.0 cluster with Ambari 2.2.2
> # Upgrade Ambari to 2.4.0.0
> # Register HDP-2.5.0.0 version and install the bits
> # Modify configs for some of the service like HDFS, ZK, YARN
> # Start EU
>
> *Result*:
> EU pre-check does *not* report below error for the three services whose
> config was modified in step 4
> "The following service configurations have been updated and their Service
> Checks should be run again:"
>
> Upon further investigation found that the pre-check does not work if a
> service check has never been run for a service at all AND reports success in
> such cases
> In other words, the comparison between last config modification time and last
> service check time succeeds if service check never ran at all and the output
> of below query returns empty:
> {code}
> SELECT start_time FROM host_role_command where role = 'HDFS_SERVICE_CHECK'
> AND status = 'COMPLETED' ORDER BY start_time DESC;
> {code}
>
> In this case when I manually ran a service check for HDFS and retried EU, the
> pre-check caught the mismatch and reported error
>
> *Note*: I believe we do run service check as part of cluster install, but
> looks like it does not get updated in the DB tables for all services.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java
> 9abfa50
>
> ambari-server/src/test/java/org/apache/ambari/server/checks/ServiceCheckValidityCheckTest.java
> 55429bd
>
> Diff: https://reviews.apache.org/r/48041/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 47978: VDF should use package-version for the os, not the release
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47978/
---
(Updated May 31, 2016, 9:20 a.m.)
Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
Bugs: AMBARI-16933
https://issues.apache.org/jira/browse/AMBARI-16933
Repository: ambari
Description
---
The aggregated VDF skips package-version as it is defined in the {{release}}
element. This is incorrect as it implies that information is consistent for
all repositories in the VDF. The element has been moved, so code needs to
account for that when formulating commands to the agent.
Luckily, code already existed for sending down the package-version, so it was
changing that code to pull for the correct osFamily.
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
f4a615c
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
532ab2f
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
4ee6670
ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java
39b30e7
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
f6863be
ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
69eb39d
ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java
237eed2
ambari-server/src/test/resources/hbase_version_test.xml 183da8c
Diff: https://reviews.apache.org/r/47978/diff/
Testing (updated)
---
Manual. Automated:
Tests run: 4370, Failures: 0, Errors: 0, Skipped: 34
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 46:24.103s
[INFO] Finished at: Fri May 27 18:08:19 EDT 2016
[INFO] Final Memory: 33M/672M
[INFO]
Thanks,
Nate Cole
Re: Review Request 47018: "ambari-server upgrade" shouldn't automatically add stack configs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47018/#review135643 --- Ship it! Ship It! - Nate Cole On May 26, 2016, 12:15 p.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47018/ > --- > > (Updated May 26, 2016, 12:15 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, > and Sumit Mohanty. > > > Bugs: AMBARI-16272 > https://issues.apache.org/jira/browse/AMBARI-16272 > > > Repository: ambari > > > Description > --- > > Today, "ambari-server upgrade" will automatically add stack configs. > However, it also causes problems when default properties or properties with > default value such as "localhost" end up being added. > > This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically > add dfs.namenode.secondary.http-address > > This logic today will even add new config types. E.g., add ranger-env even > though Ranger is not installed. If the customer then upgrades the stack from > HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs. > If we change this behavior, it's good to do so in a major release such as 2.4 > > We add required xml tags/attributes to properties: > > prop_name > prop_val > > > > > we are going to enforce developers to explicitly specify what to do during > ambari/stack upgrade when adding any new config property > If any of stack configuration files does not pass XSD schema validation (that > contains this enforcement), then ambari-server unit tests will fail. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > fb3ae69 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > 8a352bd > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > a36df7b > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 34b3ba1 > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 2e857ed > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml > 260fe65 > ambari-server/src/main/resources/configuration-schema.xsd PRE-CREATION > > ambari-server/src/main/resources/scripts/configurations-set-default-update-policy.sh > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java > a4a3108 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java > 8f53f6a > > ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java > b11c5d8 > > ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java > PRE-CREATION > script.sh PRE-CREATION > > Diff: https://reviews.apache.org/r/47018/diff/ > > > Testing > --- > > Ran unit tests. Tried ambari-upgrade and stack upgrade, seems to work well. > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 48708: Namenode start step failed during EU with RetriableException
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48708/#review137599
---
Ship it!
Ship It!
- Nate Cole
On June 14, 2016, 5:33 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48708/
> ---
>
> (Updated June 14, 2016, 5:33 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate
> Cole.
>
>
> Bugs: AMBARI-17236
> https://issues.apache.org/jira/browse/AMBARI-17236
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When starting NN during an EU, we're hitting this when trying to create HDFS
> directories:
> ```
> {
> "RemoteException": {
> "exception": "RetriableException",
> "javaClassName": "org.apache.hadoop.ipc.RetriableException",
> "message": "NameNode still not started"
> }
> }
> ```
>
> So, the heart of this issue is that, depending on topology and upgrade type,
> we might not wait for NN to be out of Safe Mode after starting. However, we
> are always creating directories, regardless of topology/upgrade:
>
> ```
> # Always run this on non-HA, or active NameNode during HA.
> if is_active_namenode:
> create_hdfs_directories()
> create_ranger_audit_hdfs_directories()
> ```
>
> NameNode, in Safe Mode, is read-only and would forbid this anyway, even if it
> didn't throw a retryable exception:
> ```
> [hdfs@c6403 root]$ hadoop fs -mkdir /foo
> mkdir: Cannot create directory /foo. Name node is in safe mode.
> ```
>
> So, it seems like we need to wait for NN to be out of Safe Mode no matter
> what.
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py
> 18e61fb
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
> 635f159
>
> Diff: https://reviews.apache.org/r/48708/diff/
>
>
> Testing
> ---
>
> PENDING
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 48041: SERVICE_CHECK Upgrade pre-check does not throw error when its expected to
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48041/#review137961
---
Hi Dmitry, what's the status of this review/patch?
- Nate Cole
On May 30, 2016, 9:30 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48041/
> ---
>
> (Updated May 30, 2016, 9:30 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-16954
> https://issues.apache.org/jira/browse/AMBARI-16954
>
>
> Repository: ambari
>
>
> Description
> ---
>
> *Steps*
> # Deploy HDP-2.4.0.0 cluster with Ambari 2.2.2
> # Upgrade Ambari to 2.4.0.0
> # Register HDP-2.5.0.0 version and install the bits
> # Modify configs for some of the service like HDFS, ZK, YARN
> # Start EU
>
> *Result*:
> EU pre-check does *not* report below error for the three services whose
> config was modified in step 4
> "The following service configurations have been updated and their Service
> Checks should be run again:"
>
> Upon further investigation found that the pre-check does not work if a
> service check has never been run for a service at all AND reports success in
> such cases
> In other words, the comparison between last config modification time and last
> service check time succeeds if service check never ran at all and the output
> of below query returns empty:
> {code}
> SELECT start_time FROM host_role_command where role = 'HDFS_SERVICE_CHECK'
> AND status = 'COMPLETED' ORDER BY start_time DESC;
> {code}
>
> In this case when I manually ran a service check for HDFS and retried EU, the
> pre-check caught the mismatch and reported error
>
> *Note*: I believe we do run service check as part of cluster install, but
> looks like it does not get updated in the DB tables for all services.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java
> 9abfa50
>
> ambari-server/src/test/java/org/apache/ambari/server/checks/ServiceCheckValidityCheckTest.java
> 55429bd
>
> Diff: https://reviews.apache.org/r/48041/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 48812: Use customized display name as version string
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48812/ --- (Updated June 16, 2016, 4:27 p.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-17282 https://issues.apache.org/jira/browse/AMBARI-17282 Repository: ambari Description --- The user can specify the display name when creating a version using VDF. In that case, use that for the repository version string as well. This does not cause problems as the version will get replaced at install time. We just need a way to provide uniqueness in the DB. Also sneaked in an ask by the FE to supply stack min/max jdk with the output. They did not want to make an extra call to get that information. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 4a061c5 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java 6b66f12 Diff: https://reviews.apache.org/r/48812/diff/ Testing (updated) --- Manual. Automated (failures from other patches): Tests in error: ServiceComponentTest.testHistoryCreation:406 » NoSuchElement ServiceComponentTest.testHistoryRemoval:516 » NoSuchElement Tests run: 4481, Failures: 0, Errors: 2, Skipped: 34 [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 46:58.287s [INFO] Finished at: Thu Jun 16 16:25:33 EDT 2016 [INFO] Final Memory: 33M/665M [INFO] Thanks, Nate Cole
Re: Review Request 48817: Hive Metastore Upgrade Fails Because Of Missing Hive Interactive Directory
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48817/#review138094
---
Ship it!
Any new tests required for these changes?
- Nate Cole
On June 16, 2016, 5:02 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48817/
> ---
>
> (Updated June 16, 2016, 5:02 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Vitalyi
> Brodetskyi.
>
>
> Bugs: AMBARI-17284
> https://issues.apache.org/jira/browse/AMBARI-17284
>
>
> Repository: ambari
>
>
> Description
> ---
>
> During an upgrade from HDP 2.3/2.4 to HDP 2.5, the following error is seen:
>
> ```
> resource_management.core.exceptions.Fail: Execution of 'cp
> --remove-destination /var/lib/ambari-agent/tmp/mysql-connector-java.jar
> /usr/hdp/current/hive-server2-hive2/lib/mysql-connector-java.jar' returned 1.
> cp: cannot create regular file
> `/usr/hdp/current/hive-server2-hive2/lib/mysql-connector-java.jar': No such
> file or directory
> ```
>
> This is because with HDP 2.5, Hive introduced a new component called
> `hive-server2-hive2`:
>
> ```
> /usr/hdp/current/hive-metastore -> /usr/hdp/2.5.0.0-1234/hive
> /usr/hdp/current/hive-server2 -> /usr/hdp/2.5.0.0-1234/hive
> /usr/hdp/current/hive-server2-hive2 -> /usr/hdp/2.5.0.0-1234/hive2
> ```
>
> Normally, this would be fine. However, we must use the `schematool` from
> `hive2` and not {{hive}}:
>
> ```
> /usr/hdp/2.5.0.0-1234/hive2/bin/schematool
> ```
>
> Since Metastore is uggraded first, it's trying to write the custom JDBC
> connector to
> `/usr/hdp/current/hive-server2-hive2/lib`
>
> But since this component doesn't exist or hasn't been `hdp-select`'d yet, it
> fails.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
> ef0fd6b
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py
> 55cddd6
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py
> 0a3d921
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
> 81207d6
>
> Diff: https://reviews.apache.org/r/48817/diff/
>
>
> Testing
> ---
>
> PENDING
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 44931: Use Version Definition value for package-version when installing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44931/ --- (Updated March 17, 2016, 8:51 a.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15447 https://issues.apache.org/jira/browse/AMBARI-15447 Repository: ambari Description --- The version defintion file defines a package version that should be used when installing packages. The current calculation cannot be considered reliable, but should stay as a contingency for when NOT using a version definition. Diffs - ambari-common/src/main/python/resource_management/libraries/script/script.py a8098a0 ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java 402a338 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java e32e2f9 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java 07e62b3 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java 1cd9c0a ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java 3398709 ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java 6bcedf5 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java dea83a1 ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 ambari-server/src/test/resources/hbase_version_test.xml 9df07ed Diff: https://reviews.apache.org/r/44931/diff/ Testing (updated) --- Manual. Automated: Results : Tests run: 3965, Failures: 0, Errors: 0, Skipped: 33 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 1:10:59.814s [INFO] Finished at: Thu Mar 17 08:26:54 EDT 2016 [INFO] Final Memory: 38M/566M [INFO] Thanks, Nate Cole
Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44986/#review124220
---
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
(lines 287 - 289)
<https://reviews.apache.org/r/44986/#comment186680>
Use {} format for log statements like so: LOG.info("Recovery configuration
set to {}", response.getRecoveryConfig());
- Nate Cole
On March 17, 2016, 7 p.m., Nahappan Somasundaram wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44986/
> ---
>
> (Updated March 17, 2016, 7 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid
> Wagle.
>
>
> Bugs: AMBARI-15474
> https://issues.apache.org/jira/browse/AMBARI-15474
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-15474: Listen for changes to auto-start configuration and send them to
> the agent during heartbeats.
>
> In HeartbeatHandler::handleHeartBeat(), send the recovery configuration to
> the agent. Instead of pulling the recovery configuration from the application
> cache, implement change events and get notified whenever there are changes to
> the recovery configuration.
>
> **Changes**:
> 1. Instead of sending the recovery configuration to the agent during every
> heartbeat (10 seconds interval), send the configuration only when changes are
> detected.
> 2. Used AmbariEventPublisher to publish the changes. RecoveryConfigHelper is
> the subscriber which listens to changes to the configuration.
> 3. RecoveryConfigHelper maintains a map of Cluster-->Host-->Timestamp which
> is the last updated timestamp of the recovery configuration. This timestamp
> is used as a token and is sent to the agent with the configuration. The agent
> provides this timestamp during each heartbeat. If the timestamp from the
> heartbeat is the same as the one on the server side, no configuration is sent
> to the agent.
> 4. Whenever changes are detected, the timestamp is invalidated. During the
> next heartbeat, the recovery configuration is read from the DB and sent to
> the agent along with the latest timestamp.
> 5. Added new unit tests to test each configuration change event.
>
>
> Diffs
> -
>
> ambari-agent/src/main/python/ambari_agent/Controller.py
> c1c16ac97cca0d2677da2e26eee7b4949a4bf15c
> ambari-agent/src/main/python/ambari_agent/Heartbeat.py
> b28644c834a2387d2fb0ad17d104224e830a0245
> ambari-agent/src/main/python/ambari_agent/RecoveryManager.py
> ed537ca6928ce376c5f3e906f7a47c3f1919e309
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeat.java
> 1430aa20cf68db071be85c4ad2ebb9acdaccecc0
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
> 3a80803724bcb6ca9e4ec875aa17c9ac1c66fbe8
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
> 617b04b5b8768ba28a3c8ecb6e5e00601f153396
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfig.java
> c2c18462c586292caaf40bdb6a25a8fe9a39d76c
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java
> 92a622117c71ddfc3bd2562c04ee525491bd6ffd
>
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java
> 78731854fe80c3d6087a73ebf00b0ffe9e287354
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ClusterConfigChangedEvent.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentInstalledEvent.java
> 2ce01236b3bb1ab4934fe456e5669fa474987426
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentRecoveryChangeEvent.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
> 19712cdb18592816f82898cd7aa08a3600c7e1f4
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java
> 2cc3d0013140c9c7d3453d7c5e05c3aa02bc5794
>
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
> 4be1c21bf41416681be836716ce38305cc3f287e
>
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
> 3a0075e3a3c90088ca6bd2d9eca48ef50db9fdb3
>
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>
Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124217
---
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
(line 461)
<https://reviews.apache.org/r/45035/#comment186677>
formatting :)
- Nate Cole
On March 18, 2016, 11:20 a.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
>
> (Updated March 18, 2016, 11:20 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
>
>
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
>
>
> Repository: ambari
>
>
> Description
> ---
>
> During an upgrade, the HDFS NameNode(s) are restarted with the
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an
> upgrade and decide to "Finalize Later".
>
> This allows the cluster to run in the upgraded state before committing to the
> upgrade. Full cluster control is returned via the Ambari web client.
>
> Administrators can then decide to restart a NameNode. Upon restarting the
> NameNode, it will produce an error that it was not started with the
> {{rollingUpgrade}} flag.
>
> It seems that as long as an upgrade has not been finalized, the NameNode(s)
> must be started with the {{rollingUpgrade}} to allow them to function
> properly.
>
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and
> green.
> - Now restart a NameNode
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
> 9ea541e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
> 4ef215c
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
> 303f3a4
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
> 07061e1
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
> 3f1a52b
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
> ed3c772
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> 1c7ff61
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
> 84bb9f3
> ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql
> ab6dc93
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
> e4c8c9c
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
> 02905ec
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
> 905802f
> ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js
> 2dceccc
> ambari-web/app/utils/ajax/ajax.js 29d0715
>
> Diff: https://reviews.apache.org/r/45035/diff/
>
>
> Testing
> ---
>
> Pending...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 44352: AMBARI-15230: Move default recovery properties from ambari.properties to cluster-env.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44352/#review123645 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java (lines 61 - 64) <https://reviews.apache.org/r/44352/#comment185876> What's this for? injector.getInstance(RecoveryConfigHelper.class) injects members. ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java (lines 112 - 116) <https://reviews.apache.org/r/44352/#comment185878> Curious: just because something is in MM, the recovery enabled flag shouldn't be affected. Now what you do with the flag *IF* MM is true is another story, but from a config standpoint, I wouldn't think you would be doing this. If it's enabled, it's enabled. - Nate Cole On March 14, 2016, 6 p.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44352/ > --- > > (Updated March 14, 2016, 6 p.m.) > > > Review request for Ambari, Jonathan Hurley, Zhe (Joe) Wang, Nate Cole, Sumit > Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15230 > https://issues.apache.org/jira/browse/AMBARI-15230 > > > Repository: ambari > > > Description > --- > > AMBARI-15230: Move default recovery properties from ambari.properties to > cluster-env.xml > > **Changes**: > > 1. Moved all *recovery* properties from ambari.properties to cluster-env.xml > 2. Moved Configuration properties for recovery from Configuration.java to a > new class ClusterEnvConfig > 3. Populate RecoveryConfig from ClusterEnvConfig instead of Configuration. > 4. Modified existing unit tests > 5. Added new unit tests > > > Diffs > - > > ambari-server/conf/unix/ambari.properties > 92dec2480025cf7e2148da7db327814ef25207c1 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > a13b4211e7dfffc678d8e8f55fa892bc71eca7e1 > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfig.java > 3da86092ffd0477ea37d73f5fa9f5de857ec0908 > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 43fdfe4fe68900f31e984e1a1681978bdd7cff98 > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java > 2a062a779a651c33190dd0aa77a560e7e38aac4b > > ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java > 98dc1b793e95b5f9612d0e5ba465227083b8e804 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml > 3fb82e9dcf5afe0af0088385ce73b53f96962940 > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java > a5a3cb5bb7639a1e9e11ed66dcf30a76152a8175 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java > e29e23ef61db6d238d0944529c2c4cbd0e227116 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/44352/diff/ > > > Testing > --- > > 1. ** mvn clean install -DskipTests ** > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [6.883s] > [INFO] Apache Ambari Project POM . SUCCESS [0.037s] > [INFO] Ambari Web SUCCESS [22.487s] > [INFO] Ambari Views .. SUCCESS [1.023s] > [INFO] Ambari Admin View . SUCCESS [5.445s] > [INFO] ambari-metrics SUCCESS [0.339s] > [INFO] Ambari Metrics Common . SUCCESS [0.418s] > [INFO] Ambari Metrics Hadoop Sink SUCCESS [1.039s] > [INFO] Ambari Metrics Flume Sink . SUCCESS [0.554s] > [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.607s] > [INFO] Ambari Metrics Storm Sink . SUCCESS [1.784s] > [INFO] Ambari Metrics Collector .. SUCCESS [6.406s] > [INFO] Ambari Metrics Monitor SUCCESS [1.590s] > [INFO] Ambari Met
Review Request 44983: Service version display should be based on Version Definition
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44983/
---
Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
Bugs: AMBARI-15473
https://issues.apache.org/jira/browse/AMBARI-15473
Repository: ambari
Description
---
Add {{stack_services}} to the repository_version endpoint, similar to the
{{services}} attribute that lists the available services.
This will return a list of all stack services as they pertain to the Version
Definition File. This is required to show the correct versions that are
available for a stack based on the repo_version.
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
c298e0a
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ManifestServiceInfo.java
PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
93ac767
ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java
f2939c7
ambari-server/src/test/resources/version_definition_test.xml 69ea581
Diff: https://reviews.apache.org/r/44983/diff/
Testing
---
Manual. Automated pending.
Thanks,
Nate Cole
Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44986/#review124222 --- Ship it! Ship It! - Nate Cole On March 17, 2016, 7 p.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44986/ > --- > > (Updated March 17, 2016, 7 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid > Wagle. > > > Bugs: AMBARI-15474 > https://issues.apache.org/jira/browse/AMBARI-15474 > > > Repository: ambari > > > Description > --- > > AMBARI-15474: Listen for changes to auto-start configuration and send them to > the agent during heartbeats. > > In HeartbeatHandler::handleHeartBeat(), send the recovery configuration to > the agent. Instead of pulling the recovery configuration from the application > cache, implement change events and get notified whenever there are changes to > the recovery configuration. > > **Changes**: > 1. Instead of sending the recovery configuration to the agent during every > heartbeat (10 seconds interval), send the configuration only when changes are > detected. > 2. Used AmbariEventPublisher to publish the changes. RecoveryConfigHelper is > the subscriber which listens to changes to the configuration. > 3. RecoveryConfigHelper maintains a map of Cluster-->Host-->Timestamp which > is the last updated timestamp of the recovery configuration. This timestamp > is used as a token and is sent to the agent with the configuration. The agent > provides this timestamp during each heartbeat. If the timestamp from the > heartbeat is the same as the one on the server side, no configuration is sent > to the agent. > 4. Whenever changes are detected, the timestamp is invalidated. During the > next heartbeat, the recovery configuration is read from the DB and sent to > the agent along with the latest timestamp. > 5. Added new unit tests to test each configuration change event. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/Controller.py > c1c16ac97cca0d2677da2e26eee7b4949a4bf15c > ambari-agent/src/main/python/ambari_agent/Heartbeat.py > b28644c834a2387d2fb0ad17d104224e830a0245 > ambari-agent/src/main/python/ambari_agent/RecoveryManager.py > ed537ca6928ce376c5f3e906f7a47c3f1919e309 > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeat.java > 1430aa20cf68db071be85c4ad2ebb9acdaccecc0 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > 3a80803724bcb6ca9e4ec875aa17c9ac1c66fbe8 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java > 617b04b5b8768ba28a3c8ecb6e5e00601f153396 > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfig.java > c2c18462c586292caaf40bdb6a25a8fe9a39d76c > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java > 92a622117c71ddfc3bd2562c04ee525491bd6ffd > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java > 78731854fe80c3d6087a73ebf00b0ffe9e287354 > > ambari-server/src/main/java/org/apache/ambari/server/events/ClusterConfigChangedEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentInstalledEvent.java > 2ce01236b3bb1ab4934fe456e5669fa474987426 > > ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentRecoveryChangeEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java > 19712cdb18592816f82898cd7aa08a3600c7e1f4 > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java > 2cc3d0013140c9c7d3453d7c5e05c3aa02bc5794 > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java > 4be1c21bf41416681be836716ce38305cc3f287e > > ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java > 3a0075e3a3c90088ca6bd2d9eca48ef50db9fdb3 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java > b1f94e38f97550123dd366984438b0bbf9c2826c > > ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java > c86b95a7cd10b0720f0a3b9676f65a8bd68dd6c9 > > ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/H
Re: Review Request 45390: Database Changes to Support Alert Repeat Tolerance
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45390/#review125714
---
Ship it!
Ship It!
- Nate Cole
On March 28, 2016, 1:21 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45390/
> ---
>
> (Updated March 28, 2016, 1:21 p.m.)
>
>
> Review request for Ambari, Nate Cole and Robert Nettleton.
>
>
> Bugs: AMBARI-15602
> https://issues.apache.org/jira/browse/AMBARI-15602
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The {{alert_definition}} table will be updated to include a nullable column
> which represents the custom repeat/retry tolerance value. This value will
> override that which is set in the {{cluster-env/alerts_repeat_tolerance}}
> property.
>
> {code}
> CREATE TABLE alert_definition (
> definition_id BIGINT NOT NULL,
> cluster_id BIGINT NOT NULL,
> ...
> repeat_tolerance SMALLINT,
> repeat_tolerance_enabled TINYINT,
> ...
> PRIMARY KEY (definition_id),
> FOREIGN KEY (cluster_id) REFERENCES clusters(cluster_id),
> CONSTRAINT uni_alert_def_name UNIQUE(cluster_id,definition_name)
> );
> {code}
>
> The scope of work includes:
> - SQL File changes
> - Entity changes
> - ResourceProvider changes to expose these values
> - UpgradeCatalog changes
> - Tests
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
> 08a708f
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity.java
> b08935c
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
> 38a3614
> ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f8c97ca
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql b306c0a
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 37744f8
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql eba1745
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql
> cad0a39
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 346af50
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e238a76
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
> 7b2d797
>
> Diff: https://reviews.apache.org/r/45390/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 45321: Ambari calculates stack downgrade as ABORTED under incorrect conditions. UI shows 'Downgrade paused' and button to resume downgrade even when progress is happening
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45321/#review125740 --- Ship it! ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java (lines 203 - 205) <https://reviews.apache.org/r/45321/#comment188591> I see this in trunk already as isSuspended() ? - Nate Cole On March 24, 2016, 6:28 p.m., Alejandro Fernandez wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45321/ > --- > > (Updated March 24, 2016, 6:28 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan > Hurley, and Nate Cole. > > > Bugs: AMBARI-15569 > https://issues.apache.org/jira/browse/AMBARI-15569 > > > Repository: ambari > > > Description > --- > > *STR:* > 1. Install Ambari 2.2.0 > 2. Upgrade stack from HDP-2.2.4 to HDP-2.3.2 > 3. At finalize step, choose downgrade option > 4. Introduce failures in some clients such as Zookeeper and Yarn. > > In this case, the status of some tasks are marked as ABORTED. > This causes the overall status of the Downgrade to be ABORTED, and hence the > UI shows an incorrect message and button. > Technically, the Downgrade is still progressing, yet the UI shows "Downgrade > paused" and a button to "Resume Downgrade" > > This means that the calculations of the stages is incorrect. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java > 1d8df9b > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > d43daca > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java > fd866a1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CalculatedStatusTest.java > 8d80fa6 > > Diff: https://reviews.apache.org/r/45321/diff/ > > > Testing > --- > > Verified on live cluster and ran unit tests in CalculateStatusTest.java > > > Thanks, > > Alejandro Fernandez > >
Re: Review Request 44265: Basic Operational Audit Logging
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44265/#review125890 --- ambari-server/src/main/java/org/apache/ambari/server/api/services/LogoutService.java (lines 43 - 46) <https://reviews.apache.org/r/44265/#comment188801> We want to stop doing this anti-pattern. @StaticallyInject is the way we handle this. ambari-server/src/main/java/org/apache/ambari/server/audit/AsyncAuditLogger.java (line 66) <https://reviews.apache.org/r/44265/#comment188805> We've seen issues in the past with unbound Queues, recommend adding a bound. ambari-server/src/main/java/org/apache/ambari/server/audit/AsyncAuditLogger.java (line 97) <https://reviews.apache.org/r/44265/#comment188806> Definitely do NOT want a non-daemon thread. Set to true. ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java (lines 63 - 64) <https://reviews.apache.org/r/44265/#comment188790> Should probably be static, and accessed via ThreadLocal. They can be expensive to make (and looks like they'll be made A LOT). ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java (line 65) <https://reviews.apache.org/r/44265/#comment188797> Can these be bound by annotation? It would make it easier for a developer to add creators then to remember to have to come here. - Nate Cole On March 24, 2016, 8:20 a.m., Daniel Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44265/ > --- > > (Updated March 24, 2016, 8:20 a.m.) > > > Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-15241 > https://issues.apache.org/jira/browse/AMBARI-15241 > > > Repository: ambari > > > Description > --- > > Entry points for logging events: > - Login and logout: AmbariAuthenticationFilter, AmbariAuthorizationFilter, > LogoutService > - Operation and tasks: ActionDBAAccessorImpl > - Kerberos related events: CreateKeytabFilesServerAction, > CreatePrincipalServerAction, DestroyPrincipalsServerAction, > FinalizeKerberosServerAction > - REST api: BaseService > > Architecture: > AuditLogger injectable is created and wired in by Guice. The default > implementation (AuditLoggerDefaultImpl) for this interface logs to a file. > Considering performance impact, there is a buffered audit logger > implementation (BufferedAuditLogger) that is a wrapper for an audit logger > implementation and does the logging asynchronously. > > Audit loggers have a single log method that accepts an AuditEvent object. At > the entry points an AuditEvent is assembled and log method is called (except > for the REST api, see below...) > > REST Api: > In BaseService, there is a RequestAuditLogger, that is responsible for > handling and assembling AuditEvents. It also has a log method, but the > parameters for that is the Request and Result objects. > RequestAuditLogger is a small framework for that plugins can be implemented > to handle different type of requests. > Plugins implements RequestAuditEventCreator interface and can be set to > handle one or multiple request types (PUT, POST, DELETE, etc...), resource > types (HOST_COMPONENT, BLUEPRINT, etc..) and results statuses (200 OK, 202 > ACCEPTED, 404 NOT_FOUND, etc). If null is returned by these getters, it means > "all". If there are more than one RequestAuditEventCreators can handle a > request, then the most specific is called. (Priority order: resourceType > > resultStatus > requestType) > For example it is possible to define a plugin for all the POST requests, but > if there is an other plugin for POST request and for resource type > HOST_COMPONENT, then this latter is used (because it is more specific). The > method createAuditEvent is responsible for creating the AuditEvent object for > the RequestAuditEventLogger. If null is returned, then request is not logged > (can be used for ignoring events). > Plugins are registered in ControllerModule and wired by guice to the > RequestAuditLogger. If a new plugin is needed, then implementing the > RequestAuditEventCreator interface and registering it in Guice is the only > things to do. (open-closed principle) > > > Diffs > - > > ambari-project/pom.xml b3d9ca2 > ambari-server/conf/unix/log4j.properties 2ee32d4 > ambari-server/conf/windows/log4j.properties cc40f74 > am
Re: Review Request 44265: Basic Operational Audit Logging
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44265/#review126138 --- Ship it! Ship It! - Nate Cole On March 30, 2016, 11:20 a.m., Daniel Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44265/ > --- > > (Updated March 30, 2016, 11:20 a.m.) > > > Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-15241 > https://issues.apache.org/jira/browse/AMBARI-15241 > > > Repository: ambari > > > Description > --- > > Entry points for logging events: > - Login and logout: AmbariAuthenticationFilter, AmbariAuthorizationFilter, > LogoutService > - Operation and tasks: ActionDBAAccessorImpl > - Kerberos related events: CreateKeytabFilesServerAction, > CreatePrincipalServerAction, DestroyPrincipalsServerAction, > FinalizeKerberosServerAction > - REST api: BaseService > > Architecture: > AuditLogger injectable is created and wired in by Guice. The default > implementation (AuditLoggerDefaultImpl) for this interface logs to a file. > Considering performance impact, there is a buffered audit logger > implementation (BufferedAuditLogger) that is a wrapper for an audit logger > implementation and does the logging asynchronously. > > Audit loggers have a single log method that accepts an AuditEvent object. At > the entry points an AuditEvent is assembled and log method is called (except > for the REST api, see below...) > > REST Api: > In BaseService, there is a RequestAuditLogger, that is responsible for > handling and assembling AuditEvents. It also has a log method, but the > parameters for that is the Request and Result objects. > RequestAuditLogger is a small framework for that plugins can be implemented > to handle different type of requests. > Plugins implements RequestAuditEventCreator interface and can be set to > handle one or multiple request types (PUT, POST, DELETE, etc...), resource > types (HOST_COMPONENT, BLUEPRINT, etc..) and results statuses (200 OK, 202 > ACCEPTED, 404 NOT_FOUND, etc). If null is returned by these getters, it means > "all". If there are more than one RequestAuditEventCreators can handle a > request, then the most specific is called. (Priority order: resourceType > > resultStatus > requestType) > For example it is possible to define a plugin for all the POST requests, but > if there is an other plugin for POST request and for resource type > HOST_COMPONENT, then this latter is used (because it is more specific). The > method createAuditEvent is responsible for creating the AuditEvent object for > the RequestAuditEventLogger. If null is returned, then request is not logged > (can be used for ignoring events). > Plugins are registered in ControllerModule and wired by guice to the > RequestAuditLogger. If a new plugin is needed, then implementing the > RequestAuditEventCreator interface and registering it in Guice is the only > things to do. (open-closed principle) > > > Diffs > - > > ambari-project/pom.xml b3d9ca2 > ambari-server/conf/unix/log4j.properties ab3ea0b > ambari-server/conf/windows/log4j.properties cc40f74 > ambari-server/pom.xml 07a315d > ambari-server/src/main/conf/log4j.properties 8e6652d > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java > a75a000 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java > a33e8a7 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java > 7945599 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/LogoutService.java > df8faaa > > ambari-server/src/main/java/org/apache/ambari/server/api/services/Request.java > ff9b6c7 > > ambari-server/src/main/java/org/apache/ambari/server/audit/AsyncAuditLogger.java > PRE-CREATION > ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLogger.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/AbstractAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apa
Re: Review Request 45347: AMBARI-15592: Auto-start services - support blueprint deployment.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45347/#review126131
---
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java
(lines 156 - 158)
<https://reviews.apache.org/r/45347/#comment189032>
getSettings() (plural) ?
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java
(lines 165 - 167)
<https://reviews.apache.org/r/45347/#comment189033>
setSettings(...) (plural) ?
- Nate Cole
On March 29, 2016, 10:17 p.m., Nahappan Somasundaram wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45347/
> ---
>
> (Updated March 29, 2016, 10:17 p.m.)
>
>
> Review request for Ambari, Ajit Kumar, Jonathan Hurley, Nate Cole, Sumit
> Mohanty, Sebastian Toader, and Sid Wagle.
>
>
> Bugs: AMBARI-15592
> https://issues.apache.org/jira/browse/AMBARI-15592
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-15592: Auto-start services - support blueprint deployment.
>
> ** Issue: **
> Define a JSON structure to specify settings for auto start in a blueprint and
> use that information to enable or disable auto start for components during
> deployment.
>
> ** Changes **
> Added a new section in the blueprint called "settings" which is a collection
> of various setting names to collection of properties. For auto start, the
> following setting names are used:
> *recovery_settings*: {"recovery_enabled" : "true" } specifies that auto start
> should be enabled for all components. Can also be specified within a
> collection, to support a uniform schema.
> *service_settings*: Collection of service names and the recovery values.
> [{"name":"HDFS", "recovery_enabled":"false"},{...}] overrides cluster level
> recovery settings.
> *component_settings*: Collection of component names and the corresponding
> recovery values. [{"name":"HDFS_CLIENT", "recovery_enabled":"true"},{...}].
> Overrides both service settings and recovery settings.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java
> 8578d6beca91bf411d0c3dafeaee42d2dd23caea
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingEntity.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
> a5e2fa1c7a2adaadc8bbe9de15b913cd9a7ca456
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java
> 11311dba0f1174248d24276a41837d7284e41607
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java
> cca28ca1529d6bccdf7a870dab7c317c1a334b1d
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
> bea036421c64b70b4bceffcbd0d13c26020a7ed1
> ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/SettingFactory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
> eb7d750702bfe83913cea92f1e1ef978ed0e6189
> ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql
> 4a0479eb3b749f332fda306ecc9b9698cde0ac92
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
> e40165d72d5d97be28882bbdf61cae5f5be67c9b
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql
> e85ea496696603b20b0efd8ffd6fb5cd2e9246ea
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
> 8bcda317c70cce320fa69127e52a786fdd372eac
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql
> c7d7cff85e1aa502976db3f7902c3d920a9e5a02
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
> 4bd7a7a9209d4d102d6d0612450280214f9b32b0
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
> 063db3aa6f4e3f14563180be2b20664f9351d852
> ambari-server/src/main/resources/META-INF/persistence.xml
> 513035f5baf6eacfcc69507069d311418546a09c
> ambari-server/src/main/resources/properties.json
> 01c15f2b1c3564c37e8203ec70f2d2b812135b13
>
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintEntityTest.java
> c660d19aee1727fd4734c07c0e5130f5bbe3c
Re: Review Request 45781: AMBARI-15722 [Ambari Web] move RedHat Satellite option out of experimental
> On April 6, 2016, 9:59 a.m., Nate Cole wrote: > > Is this hooked in anywhere? I don't see any API calls (but that's probably > > ok for this review). Any ambari-admin view changes? > > Zhe (Joe) Wang wrote: > I thought the requirement for this issue is to expose the RedHat > Satellite option, which used to be hidden within experimental flag. > I assume the feature was complete, due to > https://issues.apache.org/jira/browse/AMBARI-15047 . > And this is just a checkbox to set whether to use satellite. The API call > should be triggered at another place, taking the value of that checkbox into > account. > Yes. There is one ambari-admin view, exposing the option at version > registration. > Please let me know if my assumption was wrong. Just wanted to make sure I understood the scope of this review is all. :) Thanks! - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45781/#review127321 --- On April 5, 2016, 6:50 p.m., Zhe (Joe) Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45781/ > --- > > (Updated April 5, 2016, 6:50 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Nate Cole, Oleg > Nechiporenko, Richard Zang, Srimanth Gunturi, Xi Wang, and Yusaku Sako. > > > Bugs: AMBARI-15722 > https://issues.apache.org/jira/browse/AMBARI-15722 > > > Repository: ambari > > > Description > --- > > There is an #experimental flag that is used to control RedHat satellite > option. That needs to be moved as a regular choice in the UI and exposed. It > can be used wherever we show redhat6 urls, and applicable to redhat6/redhat7. > For example, registering repos, etc. > > > Diffs > - > > > ambari-admin/src/main/resources/ui/admin-web/app/views/stackVersions/stackVersionPage.html > 39fabf6 > ambari-web/app/templates/main/admin/stack_upgrade/edit_repositories.hbs > 62c3d14 > ambari-web/app/templates/wizard/step1.hbs e59b76d > > Diff: https://reviews.apache.org/r/45781/diff/ > > > Testing > --- > > ambari-web: > 25609 tests complete (24 seconds) > 154 tests pending > ambari-admin: > Executed 64 of 64 SUCCESS (0.077 secs / 0.299 secs) > Manual testing done. > > > Thanks, > > Zhe (Joe) Wang > >
Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services
> On March 29, 2016, 8:49 a.m., Nate Cole wrote: > > I think you need a more concrete way of ordering here. What if two > > services are marked as YARN? Which one takes precedence? You may > > want to introduce an in order to > > specifically state how it happens. Order would be a float, such that if we > > have a 1.0 and a 2.0, and some service goes in between, then we don't need > > to reorder the world (just use 1.5, or 1.1 or whatever). > > > > In addition, if you are making separate files, it's conceivable that config > > changes for services can go directly in their respective files. We broke > > them up to separate church-and-state, if you will. > > > > It seems like a more achievable goal might be to allow services to announce > > how they fit into how the upgrade packs work today, without rewriting how > > the whole system works. > > Jayush Luniya wrote: > RE: It seems like a more achievable goal might be to allow services to > announce how they fit into how the upgrade packs work today, without > rewriting how the whole system works. > +1 on allowing add-on/custom service to extend stack upgrade packs > instead of breaking down the entire upgrade pack to service level. > > Tim Thorpe wrote: > We decided we wouldn't split the upgrade xml into pieces for all the > stack services. Instead the goal will be limited to allow custom services to > integrate into the upgrade process by providing their own service upgrade xml > which will define their prereqs, order and process. This means we still need > a way for a service to indicate when during the process each of its steps > needs to be done. > > I am a bit leary of the order number because there are actually two ways > that service steps need to get added. (In the example below I will still use > core services to demonstrate). > > The first case is where a service needs to get integrated into an exists > step. In the example below, a service (HBASE) needs to be backed up prior to > upgrade: > > > KNOX > > > scripts/hbase_upgrade.py > take_snapshot > > > > > The second case is where a service has a new step which just needs to be > included in the order. In the example below, a service (KNOX) needs a new > step added: > > > KAFKA > true > > KNOX_GATEWAY > > > > Nate proposed using an order number but this gets messy in the case of > integrating into an existing step. There would be 2 order numbers, one for > the group and the other for the execute-stage: > > > > > scripts/hbase_upgrade.py > take_snapshot > > > > > This seems a bit confusing. It would also require changes to the actual > upgrade xml when now we would otherwise not require any changes to those > files. Using a tag like , does allow for some ambiguity in what the > actual order will be if several services specify that they should come after > the same step or service. I'm not convinced that it would really matter what > the order is. Yes, if a given service needs to be processed after another > then the order does matter but in that case the tag should be changed > to reflect that need. If we have two custom services that both need to be > started after HBASE then it shouldn't matter which one gets started before > the other. If there are dependencies between those two custom services then > one should state it should be started after the other. You could use order only at the group level. I wouldn't expect that you would be "injecting" execute-stage into an existing / as those are already-vetted actions. There's no reason that two groups in a row can't work against the same service/component. You're right that most cases the order doesn't matter, but the design should allow for it. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45169/#review125862 --- On March 22, 2016, 2:40 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45169/ > --- > > (Updated March 22, 2016, 2:40 p.m.) > > > Revie
Re: Review Request 45877: Add "services" element to compatible_repository_versions endpoint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45877/ --- (Updated April 7, 2016, 1:55 p.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15764 https://issues.apache.org/jira/browse/AMBARI-15764 Repository: ambari Description --- There are 3 issues fixed in this, in order of importance: * Add services/stack_services to the compatible_repository_versions endpoint. * The Repository subresource was excluding Repository objects due to a typo in the property for ClusterStackVersion * Fix for the Predicate override that should be overriding ONLY the url predicate, WITHOUT including the user predicate Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java ba3a774 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 1793ef2 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryRequest.java b674e79 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java 6221826 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java 87e73c5 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java 1622a4d ambari-server/src/test/java/org/apache/ambari/server/api/query/QueryImplTest.java 6bbb4bd ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 05f3dcf ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProviderTest.java 40d9b0b ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java 48eeaf8 Diff: https://reviews.apache.org/r/45877/diff/ Testing (updated) --- Manual. Automated: Tests run: 4121, Failures: 0, Errors: 0, Skipped: 32 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 36:36.787s [INFO] Finished at: Thu Apr 07 13:19:04 EDT 2016 [INFO] Final Memory: 35M/709M [INFO] Thanks, Nate Cole
Review Request 45877: Add "services" element to compatible_repository_versions endpoint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45877/ --- Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15764 https://issues.apache.org/jira/browse/AMBARI-15764 Repository: ambari Description --- There are 3 issues fixed in this, in order of importance: * Add services/stack_services to the compatible_repository_versions endpoint. * The Repository subresource was excluding Repository objects due to a typo in the property for ClusterStackVersion * Fix for the Predicate override that should be overriding ONLY the url predicate, WITHOUT including the user predicate Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java ba3a774 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 1793ef2 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryRequest.java b674e79 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java 6221826 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java 87e73c5 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java 1622a4d ambari-server/src/test/java/org/apache/ambari/server/api/query/QueryImplTest.java 6bbb4bd ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 05f3dcf ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProviderTest.java 40d9b0b ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java 48eeaf8 Diff: https://reviews.apache.org/r/45877/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
Re: Review Request 45877: Add "services" element to compatible_repository_versions endpoint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45877/#review127647 --- Ping - Nate Cole On April 7, 2016, 1:55 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45877/ > --- > > (Updated April 7, 2016, 1:55 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. > > > Bugs: AMBARI-15764 > https://issues.apache.org/jira/browse/AMBARI-15764 > > > Repository: ambari > > > Description > --- > > There are 3 issues fixed in this, in order of importance: > * Add services/stack_services to the compatible_repository_versions endpoint. > * The Repository subresource was excluding Repository objects due to a typo > in the property for ClusterStackVersion > * Fix for the Predicate override that should be overriding ONLY the url > predicate, WITHOUT including the user predicate > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > ba3a774 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 1793ef2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryRequest.java > b674e79 > > ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java > 6221826 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java > 87e73c5 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java > 1622a4d > > ambari-server/src/test/java/org/apache/ambari/server/api/query/QueryImplTest.java > 6bbb4bd > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > 05f3dcf > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProviderTest.java > 40d9b0b > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java > 48eeaf8 > > Diff: https://reviews.apache.org/r/45877/diff/ > > > Testing > --- > > Manual. Automated: > > Tests run: 4121, Failures: 0, Errors: 0, Skipped: 32 > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > ---- > [INFO] Total time: 36:36.787s > [INFO] Finished at: Thu Apr 07 13:19:04 EDT 2016 > [INFO] Final Memory: 35M/709M > [INFO] > > > > Thanks, > > Nate Cole > >
Re: Review Request 46100: Support option to not create a version definition resource but return the structured JSON only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46100/ --- (Updated April 12, 2016, 12:55 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Jayush Luniya. Bugs: AMBARI-15837 https://issues.apache.org/jira/browse/AMBARI-15837 Repository: ambari Description --- Adds a dry_run option to creation of VDF and return the same response as a GET. This is different from the rest of our API since part of the response comes from subresources. Subresources are only asked on GET, but there is no entity to reference in this case, so the POST/dry_run=true must contain all of it without writing to the DB. Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java ba4491e ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 963265b ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java 5a63606 Diff: https://reviews.apache.org/r/46100/diff/ Testing (updated) --- Manual. Automated: (Test in error is invalid cert against https://hortonworks.com and unrelated to this patch) Results : Tests in error: AmbariManagementControllerTest.testUpdateRepoUrlController:8524 » IllegalArgument Tests run: 4134, Failures: 0, Errors: 1, Skipped: 32 [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 35:44.990s [INFO] Finished at: Tue Apr 12 12:07:01 EDT 2016 [INFO] Final Memory: 36M/670M [INFO] Thanks, Nate Cole
Re: Review Request 45896: Atlas Integration : Support Atlas HA
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45896/#review127780 --- Ship it! Look at you, back in the mix :) - Nate Cole On April 7, 2016, 5:35 p.m., Tom Beerbower wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45896/ > --- > > (Updated April 7, 2016, 5:35 p.m.) > > > Review request for Ambari, John Speidel and Nate Cole. > > > Bugs: AMBARI-15733 > https://issues.apache.org/jira/browse/AMBARI-15733 > > > Repository: ambari > > > Description > --- > > 1 set cardinality to 1+ in Atlas stack definition > 2 expose Atlas configuration to enable HA (atlas.server.ha.enabled) > 3 set Atlas configuration for server id list (atlas.server.ids) > 4 set Atlas configuration properties for hosts (atlas.server.host.id1, ...) > 5 expose Atlas host component ha_state property in Ambari REST API. > 6 Update Ambari quick link definition in Atlas stack definition to show > ha_state. (See Yarn RM quick links) > > * Also refactored HTTPProxyPropertyProvider to remove RM specific code. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > b77fda2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AtlasServerHttpPropertyRequest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java > 11db913 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HttpPropertyProvider.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HttpProxyPropertyProvider.java > e92536c > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/JsonHttpPropertyRequest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ResourceManagerHttpPropertyRequest.java > PRE-CREATION > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py > b377757 > > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml > 7061d6b > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AtlasServerHttpPropertyRequestTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HttpPropertyProviderTest.java > b622728 > > Diff: https://reviews.apache.org/r/45896/diff/ > > > Testing > --- > > Manual test. > > mvn clean test > > new unit tests added > > > Thanks, > > Tom Beerbower > >
Review Request 46100: Support option to not create a version definition resource but return the structured JSON only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46100/ --- Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Jayush Luniya. Bugs: AMBARI-15837 https://issues.apache.org/jira/browse/AMBARI-15837 Repository: ambari Description --- Adds a dry_run option to creation of VDF and return the same response as a GET. This is different from the rest of our API since part of the response comes from subresources. Subresources are only asked on GET, but there is no entity to reference in this case, so the POST/dry_run=true must contain all of it without writing to the DB. Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java ba4491e ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 963265b ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java 5a63606 Diff: https://reviews.apache.org/r/46100/diff/ Testing --- Manual. Automated in progress. Thanks, Nate Cole
Re: Review Request 45811: Compatible Stacks not returning correctly
> On April 6, 2016, 12:06 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java, > > line 75 > > <https://reviews.apache.org/r/45811/diff/1/?file=1327547#file1327547line75> > > > > Does this also need "CompatibleRepositoryVersions/"? > > If not, ship it. No, it's a marker field only and not meant to be included in the response. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45811/#review127353 --- On April 6, 2016, 10:03 a.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45811/ > --- > > (Updated April 6, 2016, 10:03 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Jayush > Luniya. > > > Bugs: AMBARI-15717 > https://issues.apache.org/jira/browse/AMBARI-15717 > > > Repository: ambari > > > Description > --- > > A previous bug was allowing compatible stacks to come across correctly. > After that fix, that made the compatible_repository_version API filter > responses that it shouldn't have. This led to an interesting problem whereby > we needed a mechanism to alter the incoming predicate to allow those > resources to be included. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > 36ed189 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterControllerImpl.java > 9f864f8 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersion.java > 1c5fe89 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java > 26b9645 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ReadOnlyResourceProvider.java > a72527d > > ambari-server/src/main/java/org/apache/ambari/server/controller/spi/ClusterController.java > aa1f09f > > ambari-server/src/test/java/org/apache/ambari/server/api/query/QueryImplTest.java > f869ba3 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProviderTest.java > faef1c9 > > Diff: https://reviews.apache.org/r/45811/diff/ > > > Testing > --- > > Manual. Automated pending. > > > Thanks, > > Nate Cole > >
Re: Review Request 45781: AMBARI-15722 [Ambari Web] move RedHat Satellite option out of experimental
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45781/#review127321 --- Ship it! Is this hooked in anywhere? I don't see any API calls (but that's probably ok for this review). Any ambari-admin view changes? - Nate Cole On April 5, 2016, 6:50 p.m., Zhe (Joe) Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45781/ > --- > > (Updated April 5, 2016, 6:50 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Nate Cole, Oleg > Nechiporenko, Richard Zang, Srimanth Gunturi, Xi Wang, and Yusaku Sako. > > > Bugs: AMBARI-15722 > https://issues.apache.org/jira/browse/AMBARI-15722 > > > Repository: ambari > > > Description > --- > > There is an #experimental flag that is used to control RedHat satellite > option. That needs to be moved as a regular choice in the UI and exposed. It > can be used wherever we show redhat6 urls, and applicable to redhat6/redhat7. > For example, registering repos, etc. > > > Diffs > - > > > ambari-admin/src/main/resources/ui/admin-web/app/views/stackVersions/stackVersionPage.html > 39fabf6 > ambari-web/app/templates/main/admin/stack_upgrade/edit_repositories.hbs > 62c3d14 > ambari-web/app/templates/wizard/step1.hbs e59b76d > > Diff: https://reviews.apache.org/r/45781/diff/ > > > Testing > --- > > ambari-web: > 25609 tests complete (24 seconds) > 154 tests pending > ambari-admin: > Executed 64 of 64 SUCCESS (0.077 secs / 0.299 secs) > Manual testing done. > > > Thanks, > > Zhe (Joe) Wang > >
Re: Review Request 45903: AMBARI-15775 Integrate Red Hat Satellite option in Ambari Admin
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45903/#review127818
---
Ship it!
Ship It!
- Nate Cole
On April 7, 2016, 8:36 p.m., Zhe (Joe) Wang wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45903/
> ---
>
> (Updated April 7, 2016, 8:36 p.m.)
>
>
> Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Nate Cole, Oleg
> Nechiporenko, Richard Zang, Srimanth Gunturi, Xi Wang, and Yusaku Sako.
>
>
> Bugs: AMBARI-15775
> https://issues.apache.org/jira/browse/AMBARI-15775
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Provide RedHat satellite option. It can be used wherever we show redhat6
> urls, and applicable to redhat6/redhat7. For example, registering repos, etc.
> The API call to update links when using VDF can specify the new property. In
> order to make it more generic (and default true), it is called
> "ambari_managed_repositories". API looks like:
> PUT /api/v1/stacks/HDP/versions/2.3/repository_versions/2
> {
> "RepositoryVersions" : {
> "id" : 2
> },
> "operating_systems" : [
> {
> "OperatingSystems" : {
> "ambari_managed_repositories": false,
> "os_type" : "redhat6",
> "repository_version_id" : 2,
> "stack_name" : "HDP",
> "stack_version" : "2.3"
> },
> "repositories" : [
> {
> "Repositories" : {
> "base_url" : "http://repos.ambari.apache.org/hdp/HDP-2.3.4.14-4;,
> "repo_id" : "HDP-2.3",
> "repo_name" : "HDP"
> }
> },
> {
> "Repositories" : {
> "base_url" :
> "http://repos.ambari.apache.org/hdp/HDP-UTILS-1.1.0.20;,
> "repo_id" : "HDP-UTILS-1.1.0.20",
> "repo_name" : "HDP-UTILS"
> }
> }
> ]
> }
> ]
> }
>
>
> Diffs
> -
>
>
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsCreateCtrl.js
> 6feeeac
>
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsEditCtrl.js
> c209fcc
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Stack.js
> 4ba0fc1
>
> ambari-admin/src/main/resources/ui/admin-web/app/views/stackVersions/stackVersionPage.html
> 39fabf6
>
> Diff: https://reviews.apache.org/r/45903/diff/
>
>
> Testing
> ---
>
> ambari-admin:
> Executed 64 of 64 SUCCESS (0.048 secs / 0.299 secs)
> Manual testing done.
>
>
> Thanks,
>
> Zhe (Joe) Wang
>
>
Re: Review Request 46046: Not all operations shown on 'Background Operations' window
> On April 11, 2016, 3:33 p.m., Jonathan Hurley wrote: > > So with this change, we're basically removing the ability to get back all > > cluster and host requests in the same query, right? I think that's OK. Correct. We could make the /requests (no cluster) one include the cluster-based ones, but it's not working that way today anyway so it's not a real change in functionality. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46046/#review128218 --- On April 11, 2016, 3:27 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46046/ > --- > > (Updated April 11, 2016, 3:27 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Sumit > Mohanty. > > > Bugs: AMBARI-15810 > https://issues.apache.org/jira/browse/AMBARI-15810 > > > Repository: ambari > > > Description > --- > > The API is filtering Request objects in an inconsistent manner. When > accessing /clusters/c1/requests, the db query includes non-cluster calls, > which get filtered out. The UI then sees the count returned less than the > count asked for, and does not allow any more queries. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestResourceProvider.java > c1e9053 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java > dcbf2a6 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java > bbb55f3 > > Diff: https://reviews.apache.org/r/46046/diff/ > > > Testing > --- > > Manual. Automated: > > Tests run: 4132, Failures: 0, Errors: 0, Skipped: 32 > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 36:41.820s > [INFO] Finished at: Mon Apr 11 14:17:50 EDT 2016 > [INFO] Final Memory: 34M/766M > [INFO] > > > > Thanks, > > Nate Cole > >
Review Request 44931: Use Version Definition value for package-version when installing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44931/ --- Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15447 https://issues.apache.org/jira/browse/AMBARI-15447 Repository: ambari Description --- The version defintion file defines a package version that should be used when installing packages. The current calculation cannot be considered reliable, but should stay as a contingency for when NOT using a version definition. Diffs - ambari-common/src/main/python/resource_management/libraries/script/script.py a8098a0 ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java 402a338 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java e32e2f9 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java 07e62b3 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java 1cd9c0a ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java 3398709 ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java 6bcedf5 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java dea83a1 ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 ambari-server/src/test/resources/hbase_version_test.xml 9df07ed Diff: https://reviews.apache.org/r/44931/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
Re: Review Request 44931: Use Version Definition value for package-version when installing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44931/ --- (Updated March 16, 2016, 6:08 p.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15447 https://issues.apache.org/jira/browse/AMBARI-15447 Repository: ambari Description --- The version defintion file defines a package version that should be used when installing packages. The current calculation cannot be considered reliable, but should stay as a contingency for when NOT using a version definition. Diffs - ambari-common/src/main/python/resource_management/libraries/script/script.py a8098a0 ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java 402a338 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java e32e2f9 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java 07e62b3 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java 1cd9c0a ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java 3398709 ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java 6bcedf5 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java dea83a1 ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 ambari-server/src/test/resources/hbase_version_test.xml 9df07ed Diff: https://reviews.apache.org/r/44931/diff/ Testing (updated) --- Manual. Automated: Tests run: 3961, Failures: 0, Errors: 0, Skipped: 33 [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 43:00.985s [INFO] Finished at: Wed Mar 16 17:50:32 EDT 2016 [INFO] Final Memory: 35M/776M [INFO] Thanks, Nate Cole
Re: Review Request 44878: Atlas Integration : Rename Atlas Configurations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44878/#review123865 --- Ship it! Ship It! - Nate Cole On March 15, 2016, 8:57 p.m., Tom Beerbower wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44878/ > --- > > (Updated March 15, 2016, 8:57 p.m.) > > > Review request for Ambari, John Speidel, Nate Cole, and Sumit Mohanty. > > > Bugs: AMBARI-15431 > https://issues.apache.org/jira/browse/AMBARI-15431 > > > Repository: ambari > > > Description > --- > > Atlas configuration name application.properties has been changed to > atlas-application.properties to avoid name conflicts with other services. > See https://issues.apache.org/jira/browse/ATLAS-392. > > Ambari scripts for Atlas currently use the configuration name > application.properties for all stack levels. Stacks which include Atlas > 0.5 > should use the configuration name atlas-application.properties. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml > 8500488 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py > 6df47b0 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py > 5a39278 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py > 38c2c9b > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py > c402721 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py > b131574 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py > 115ed4f > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py > ec37243 > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py > da67775 > > ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-env.xml > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml > 9514576 > ambari-server/src/test/python/stacks/2.3/configs/default.json d5102dc > ambari-server/src/test/python/stacks/2.3/configs/secure.json 2b60be6 > ambari-server/src/test/python/stacks/2.6/ATLAS/test_atlas_server.py > PRE-CREATION > ambari-server/src/test/python/stacks/2.6/configs/default.json PRE-CREATION > > Diff: https://reviews.apache.org/r/44878/diff/ > > > Testing > --- > > Manual test install Atlas (HDP 2.6 stack). Verify configuration. > > mvn clean test > > all pass > > > Thanks, > > Tom Beerbower > >
Re: Review Request 44983: Service version display should be based on Version Definition
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44983/
---
(Updated March 18, 2016, 10:43 a.m.)
Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
Bugs: AMBARI-15473
https://issues.apache.org/jira/browse/AMBARI-15473
Repository: ambari
Description
---
Add {{stack_services}} to the repository_version endpoint, similar to the
{{services}} attribute that lists the available services.
This will return a list of all stack services as they pertain to the Version
Definition File. This is required to show the correct versions that are
available for a stack based on the repo_version.
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
c298e0a
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ManifestServiceInfo.java
PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
93ac767
ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java
f2939c7
ambari-server/src/test/resources/version_definition_test.xml 69ea581
Diff: https://reviews.apache.org/r/44983/diff/
Testing (updated)
---
Manual. Automated:
Tests run: 3963, Failures: 0, Errors: 0, Skipped: 33
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 35:40.607s
[INFO] Finished at: Fri Mar 18 10:34:17 EDT 2016
[INFO] Final Memory: 37M/770M
[INFO]
Thanks,
Nate Cole
Re: Review Request 45218: Atlas Integration : Rename Atlas Configurations (2.5 stack definition)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45218/#review125063 --- Ship it! Ship It! - Nate Cole On March 23, 2016, 12:07 p.m., Tom Beerbower wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45218/ > --- > > (Updated March 23, 2016, 12:07 p.m.) > > > Review request for Ambari, John Speidel, Nate Cole, and Sumit Mohanty. > > > Bugs: AMBARI-15431 > https://issues.apache.org/jira/browse/AMBARI-15431 > > > Repository: ambari > > > Description > --- > > Move changes for AMBARI-15431 from 2.6 stack definition to 2.5 stack > definition. > > Atlas configuration name application.properties has been changed to > atlas-application.properties to avoid name conflicts with other services. > See https://issues.apache.org/jira/browse/ATLAS-392. > > Ambari scripts for Atlas currently use the configuration name > application.properties for all stack levels. Stacks which include Atlas > 0.5 > should use the configuration name atlas-application.properties. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/atlas-env.xml > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml > 66aea9d > > ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-env.xml > 42503b5 > ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml > af1a047 > ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py > PRE-CREATION > ambari-server/src/test/python/stacks/2.5/configs/default.json PRE-CREATION > ambari-server/src/test/python/stacks/2.6/ATLAS/test_atlas_server.py 8e51ea0 > ambari-server/src/test/python/stacks/2.6/configs/default.json 2e1bc68 > > Diff: https://reviews.apache.org/r/45218/diff/ > > > Testing > --- > > Manual test install Atlas (HDP 2.5 stack). Verify configuration. > > mvn clean test > > all pass > > > Thanks, > > Tom Beerbower > >
Re: Review Request 45253: AMBARI-15544: Creating multinode cluster using Blueprints fails.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45253/#review125295 --- Ship it! Ship It! - Nate Cole On March 24, 2016, 12:42 p.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45253/ > --- > > (Updated March 24, 2016, 12:42 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, > Sebastian Toader, and Sid Wagle. > > > Bugs: AMBARI-15544 > https://issues.apache.org/jira/browse/AMBARI-15544 > > > Repository: ambari > > > Description > --- > > AMBARI-15544: Creating multinode cluster using Blueprints fails. > > ** Issue **: > > This issue happens when there are multiple agents running before deployment > happens. During registration, there is no cluster, so the recovery > configuration is not obtained. Subsequently, when hosts become a part of a > cluster, agents attempt to get the recovery configuration during the > heartbeats. The first agent successfully gets the configuration because the > timestamp map is empty. When the next agent heartbeats, it checks to see if > the configuration is stale. While there is an entry for the cluster name in > the timestamp map created by the previous agent, there is no hostname entry > for the current agent which causes the timestamp returned for that hostname > to be null. > > ** Fix **: > > Check the returned Timestamp object for null before accessing it. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java > dca4a9b9a32377a2d7d620d6f939e7250cc40590 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java > f7c85b6d514cee908f07513e62874ed9ed393fa4 > > Diff: https://reviews.apache.org/r/45253/diff/ > > > Testing > --- > > ** 1. mvn clean install -DskipTests ** > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [5.199s] > [INFO] Apache Ambari Project POM . SUCCESS [0.037s] > [INFO] Ambari Web SUCCESS [31.250s] > [INFO] Ambari Views .. SUCCESS [1.140s] > [INFO] Ambari Admin View . SUCCESS [5.628s] > [INFO] ambari-metrics SUCCESS [0.355s] > [INFO] Ambari Metrics Common . SUCCESS [0.476s] > [INFO] Ambari Metrics Hadoop Sink SUCCESS [1.067s] > [INFO] Ambari Metrics Flume Sink . SUCCESS [0.562s] > [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.595s] > [INFO] Ambari Metrics Storm Sink . SUCCESS [1.437s] > [INFO] Ambari Metrics Collector .. SUCCESS [6.724s] > [INFO] Ambari Metrics Monitor SUCCESS [2.089s] > [INFO] Ambari Metrics Grafana SUCCESS [0.862s] > [INFO] Ambari Metrics Assembly ... SUCCESS [1:17.652s] > [INFO] Ambari Server . SUCCESS [2:31.754s] > [INFO] Ambari Functional Tests ... SUCCESS [1.281s] > [INFO] Ambari Agent .. SUCCESS [22.491s] > [INFO] Ambari Client . SUCCESS [0.061s] > [INFO] Ambari Python Client .. SUCCESS [1.008s] > [INFO] Ambari Groovy Client .. SUCCESS [2.175s] > [INFO] Ambari Shell .. SUCCESS [0.058s] > [INFO] Ambari Python Shell ... SUCCESS [0.694s] > [INFO] Ambari Groovy Shell ... SUCCESS [1.028s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 5:16.311s > [INFO] Finished at: Wed Mar 23 15:30:45 PDT 2016 > [INFO] Final Memory: 261M/1167M > [INFO] > > > ** 2. Manual tests ** > > Deployed a cluster with 3 nodes
Re: Review Request 45302: Use repository version number to indicate repository to use when installing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45302/ --- (Updated March 27, 2016, 9:03 a.m.) Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Jonathan Hurley. Bugs: AMBARI-15564 https://issues.apache.org/jira/browse/AMBARI-15564 Repository: ambari Description --- Add a way to supply the repo_version to use when creating a cluster. This will result in a new cluster_version record BEFORE any hosts have been added to the cluster. This work effectively unblocks the UI to be able to supply a version when creating the cluster. Including Blueprints. What this patch does NOT do (other patches will address this): - Refactor how RepositoryVersionState is computed for installs to work just like upgrades. - Create a cluster using a Version Definition File (VDF) url, pull down the file, and create the repo_version record. To use this feature, the repo_version must be created BEFORE supplying the version with the cluster. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 32da5e8 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 23d43aa ambari-server/src/main/java/org/apache/ambari/server/controller/ClusterRequest.java 5c7548c ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f7d359c ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a1740fb ambari-server/src/main/java/org/apache/ambari/server/controller/predicate/ComparisonPredicate.java cc8cfdb ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java ddd07f9 ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryVersionState.java 119205a ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 8d6fec1 ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java 87225ad ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java c317162 ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java ec38f22 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java 57cbebc ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java 1613d11 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java 6e7c975 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 91f4993 Diff: https://reviews.apache.org/r/45302/diff/ Testing (updated) --- Manual. Automated: Tests run: 3992, Failures: 0, Errors: 0, Skipped: 33 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 33:21.378s [INFO] Finished at: Thu Mar 24 15:59:06 EDT 2016 [INFO] Final Memory: 33M/610M [INFO] Thanks, Nate Cole
Re: Review Request 45253: AMBARI-15544: Creating multinode cluster using Blueprints fails.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45253/#review125232 --- Ship it! ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java (lines 150 - 152) <https://reviews.apache.org/r/45253/#comment188022> Is there a unit test that can be added to make sure we don't accidentally revert this fix somehow? - Nate Cole On March 23, 2016, 6:45 p.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45253/ > --- > > (Updated March 23, 2016, 6:45 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, > Sebastian Toader, and Sid Wagle. > > > Bugs: AMBARI-15544 > https://issues.apache.org/jira/browse/AMBARI-15544 > > > Repository: ambari > > > Description > --- > > AMBARI-15544: Creating multinode cluster using Blueprints fails. > > ** Issue **: > > This issue happens when there are multiple agents running before deployment > happens. During registration, there is no cluster, so the recovery > configuration is not obtained. Subsequently, when hosts become a part of a > cluster, agents attempt to get the recovery configuration during the > heartbeats. The first agent successfully gets the configuration because the > timestamp map is empty. When the next agent heartbeats, it checks to see if > the configuration is stale. While there is an entry for the cluster name in > the timestamp map created by the previous agent, there is no hostname entry > for the current agent which causes the timestamp returned for that hostname > to be null. > > ** Fix **: > > Check the returned Timestamp object for null before accessing it. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java > dca4a9b9a32377a2d7d620d6f939e7250cc40590 > > Diff: https://reviews.apache.org/r/45253/diff/ > > > Testing > --- > > ** 1. mvn clean install -DskipTests ** > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [5.199s] > [INFO] Apache Ambari Project POM . SUCCESS [0.037s] > [INFO] Ambari Web SUCCESS [31.250s] > [INFO] Ambari Views .. SUCCESS [1.140s] > [INFO] Ambari Admin View . SUCCESS [5.628s] > [INFO] ambari-metrics SUCCESS [0.355s] > [INFO] Ambari Metrics Common . SUCCESS [0.476s] > [INFO] Ambari Metrics Hadoop Sink SUCCESS [1.067s] > [INFO] Ambari Metrics Flume Sink . SUCCESS [0.562s] > [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.595s] > [INFO] Ambari Metrics Storm Sink . SUCCESS [1.437s] > [INFO] Ambari Metrics Collector .. SUCCESS [6.724s] > [INFO] Ambari Metrics Monitor SUCCESS [2.089s] > [INFO] Ambari Metrics Grafana SUCCESS [0.862s] > [INFO] Ambari Metrics Assembly ... SUCCESS [1:17.652s] > [INFO] Ambari Server . SUCCESS [2:31.754s] > [INFO] Ambari Functional Tests ... SUCCESS [1.281s] > [INFO] Ambari Agent .. SUCCESS [22.491s] > [INFO] Ambari Client . SUCCESS [0.061s] > [INFO] Ambari Python Client .. SUCCESS [1.008s] > [INFO] Ambari Groovy Client .. SUCCESS [2.175s] > [INFO] Ambari Shell .. SUCCESS [0.058s] > [INFO] Ambari Python Shell ... SUCCESS [0.694s] > [INFO] Ambari Groovy Shell ... SUCCESS [1.028s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 5:16.311s > [INFO] Finished at: Wed Mar 23 15:30:45 PDT 2016 > [INFO] Final Memory: 261M/1167M > [INFO] > ---
Re: Review Request 45586: Provide an in-memory representation of available VDF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45586/ --- (Updated April 1, 2016, 10:52 a.m.) Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15652 https://issues.apache.org/jira/browse/AMBARI-15652 Repository: ambari Description --- * Load additional information out of hdp_urlinfo.json that will load VDF from a known location * Merging these separate VDF into one representation due to Ambari structures * Expose these available VDF via endpoint (included using established OperatingSystem and Repository sub-resources) * Allow creation of internal repo_version based on an available VDF Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java 67d9439 ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 0f09d94 ambari-server/src/main/java/org/apache/ambari/server/api/services/VersionDefinitionService.java e637850 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d1f8232 ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemRequest.java 16136db ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemResponse.java cc9bb70 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java 63f6b60 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperatingSystemResourceProvider.java b982583 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java a37364f ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 3ab5169 ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 0d87b68 ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 50bc30f ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 0ad24fe ambari-server/src/main/resources/version_definition.xsd 3c0399f ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java d729d2a ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 58f00e9 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java efdf84e ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java 707057f ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json 37a6a60 ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.4-123.xml PRE-CREATION contrib/version-builder/version_builder.py f209894 Diff: https://reviews.apache.org/r/45586/diff/ Testing (updated) --- Manual. Automated: Tests run: 4096, Failures: 0, Errors: 0, Skipped: 33 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 35:07.392s [INFO] Finished at: Fri Apr 01 09:35:30 EDT 2016 [INFO] Final Memory: 37M/754M [INFO] Thanks, Nate Cole
Re: Review Request 45586: Provide an in-memory representation of available VDF
> On April 1, 2016, 2:11 p.m., Alejandro Fernandez wrote: > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java, > > line 251 > > <https://reviews.apache.org/r/45586/diff/2/?file=1322455#file1322455line251> > > > > Do both of these tests require internet connection and take up to 45 > > secs each? That was just to make sure the load thread finishes before the test does (which makes all the assertions randomly false). Tests don't access the internet, they all use files (double checked, but didn't write that code). Will bump it down to 10s or so. > On April 1, 2016, 2:11 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java, > > line 94 > > <https://reviews.apache.org/r/45586/diff/2/?file=1322448#file1322448line94> > > > > is show_available supposed to be a top-level param? Will fix. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45586/#review126619 --- On April 1, 2016, 12:43 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45586/ > --- > > (Updated April 1, 2016, 12:43 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and > Jonathan Hurley. > > > Bugs: AMBARI-15652 > https://issues.apache.org/jira/browse/AMBARI-15652 > > > Repository: ambari > > > Description > --- > > * Load additional information out of hdp_urlinfo.json that will load VDF from > a known location > * Merging these separate VDF into one representation due to Ambari structures > * Expose these available VDF via endpoint (included using established > OperatingSystem and Repository sub-resources) > * Allow creation of internal repo_version based on an available VDF > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java > 67d9439 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > 0f09d94 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/VersionDefinitionService.java > e637850 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > d1f8232 > > ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemRequest.java > 16136db > > ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemResponse.java > cc9bb70 > > ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java > 63f6b60 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperatingSystemResourceProvider.java > b982583 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java > a37364f > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java > 3ab5169 > ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java > 0d87b68 > > ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java > 50bc30f > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java > 0ad24fe > ambari-server/src/main/resources/version_definition.xsd 3c0399f > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > d729d2a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 58f00e9 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java > efdf84e > > ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java > 707057f > ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json 37a6a60 > > ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.4-123.xml > PRE-CREATION > contrib/version-builder/version_builder.py f209894 > > Diff: https://reviews.apache.org/r/45586/diff/ > > > Testing > --- > > Manual. Automated: &
Re: Review Request 45586: Provide an in-memory representation of available VDF
> On April 1, 2016, 12:18 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java, > > line 230 > > <https://reviews.apache.org/r/45586/diff/1/?file=1322275#file1322275line230> > > > > Does the schema factory close the stream for you? Good point. I'll make sure of it. > On April 1, 2016, 12:18 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/resources/version_definition.xsd, line 33 > > <https://reviews.apache.org/r/45586/diff/1/?file=1322277#file1322277line33> > > > > Should it be dumped since we never use it? Do we even care to validate > > it or read it in? Thanks for reviewing! Registering repos for upgrade still uses it since those can still be per-os, even if the use case here means it's optional. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45586/#review126595 --- On April 1, 2016, 11:59 a.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45586/ > --- > > (Updated April 1, 2016, 11:59 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and > Jonathan Hurley. > > > Bugs: AMBARI-15652 > https://issues.apache.org/jira/browse/AMBARI-15652 > > > Repository: ambari > > > Description > --- > > * Load additional information out of hdp_urlinfo.json that will load VDF from > a known location > * Merging these separate VDF into one representation due to Ambari structures > * Expose these available VDF via endpoint (included using established > OperatingSystem and Repository sub-resources) > * Allow creation of internal repo_version based on an available VDF > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java > 67d9439 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > 0f09d94 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/VersionDefinitionService.java > e637850 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > d1f8232 > > ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemRequest.java > 16136db > > ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemResponse.java > cc9bb70 > > ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java > 63f6b60 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperatingSystemResourceProvider.java > b982583 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java > a37364f > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java > 3ab5169 > ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java > 0d87b68 > > ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java > 50bc30f > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java > 0ad24fe > ambari-server/src/main/resources/version_definition.xsd 3c0399f > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > d729d2a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 58f00e9 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java > efdf84e > > ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java > 707057f > ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json 37a6a60 > > ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.4-123.xml > PRE-CREATION > contrib/version-builder/version_builder.py f209894 > > Diff: https://reviews.apache.org/r/45586/diff/ > > > Testing > --- > > Manual. Automated: > > > Tests run: 4096, Failures: 0, Errors: 0, Skipped: 33 > > [INFO] > ---- > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 35:07.392s > [INFO] Finished at: Fri Apr 01 09:35:30 EDT 2016 > [INFO] Final Memory: 37M/754M > [INFO] > > > > Thanks, > > Nate Cole > >
Re: Review Request 45703: Allow skipping creation of repo files
> On April 4, 2016, 10:26 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py, > > lines 121-122 > > <https://reviews.apache.org/r/45703/diff/1/?file=1324948#file1324948line121> > > > > Is this really a warning? Kind of odd to warn on a feature. Will fix. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45703/#review127013 --- On April 4, 2016, 5 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45703/ > --- > > (Updated April 4, 2016, 5 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and > Jonathan Hurley. > > > Bugs: AMBARI-15697 > https://issues.apache.org/jira/browse/AMBARI-15697 > > > Repository: ambari > > > Description > --- > > RH Satellite requires that Ambari not manage the repositories for an OS. Add > an attribute to OperatingSystems that gets persisted to do this work. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > f3197cb > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > a0b98f7 > > ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemResponse.java > 06b4148 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java > bb50820 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperatingSystemResourceProvider.java > fd4a91f > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/OperatingSystemEntity.java > 3c881a1 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java > 6a36522 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > c01dbb3 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 05751fa > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 4da8896 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java > e031fc8 > ambari-server/src/test/python/custom_actions/TestInstallPackages.py ad4206b > > ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py > aacd1f2 > > Diff: https://reviews.apache.org/r/45703/diff/ > > > Testing > --- > > Manual. Automated pending > > > Thanks, > > Nate Cole > >
Re: Review Request 45442: Orphaned Host Alerts Cause Stale Alert Notifications After Removing Hosts
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45442/#review125940
---
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
(lines 404 - 405)
<https://reviews.apache.org/r/45442/#comment188868>
Would anyone need any other detail? There's a lot to trigger this: "Unable
to process alert for ... due to ..." Also, is it really in-error, or warning?
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
(lines 392 - 397)
<https://reviews.apache.org/r/45442/#comment188869>
I have no idea how this hostClusterMap relationship came to be :)
- Nate Cole
On March 29, 2016, 3:32 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45442/
> ---
>
> (Updated March 29, 2016, 3:32 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate
> Cole.
>
>
> Bugs: AMBARI-15620
> https://issues.apache.org/jira/browse/AMBARI-15620
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Host-level alerts from {{AMBARI}}/{{AMBARI_AGENT}} are orphaned after
> removing a host because they are always considered valid.
>
> STR
> - Deploy cluster
> - Add/Remove nodes a few times
> - Removed all aded nodes
>
> {code}
> There are 4 stale alerts from 4 host(s):
> amb-roll-workflow1458640758-5.novalocal[Host Disk Usage (3h 52m)],
> amb-roll-workflow1458640758-2.novalocal[Host Disk Usage (3h 52m)],
> amb-roll-workflow1458640758-3.novalocal[Host Disk Usage (3h 52m)],
> amb-roll-workflow1458640758-4.novalocal[Host Disk Usage (3h 52m)]
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
> 8dc8e1e
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostDAO.java
> ebd29e3
> ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java
> a1ebaba
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
> 6c68d0e
>
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java
> 136a756
>
> Diff: https://reviews.apache.org/r/45442/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 45487: VDF creation script should be callable as an importable module
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45487/ --- Review request for Ambari and Jonathan Hurley. Bugs: AMBARI-15631 https://issues.apache.org/jira/browse/AMBARI-15631 Repository: ambari Description --- The version_builder.py script should be a callable module Diffs - contrib/version-builder/example.py PRE-CREATION contrib/version-builder/example.sh 7016997 contrib/version-builder/version_builder.py 6c20a47 Diff: https://reviews.apache.org/r/45487/diff/ Testing --- Manual. No automated tests for contrib. Thanks, Nate Cole
Re: Review Request 45487: VDF creation script should be callable as an importable module
> On March 30, 2016, 10:21 a.m., Jonathan Hurley wrote: > > contrib/version-builder/version_builder.py, line 231 > > <https://reviews.apache.org/r/45487/diff/1/?file=1319578#file1319578line231> > > > > Should this be a member of the class; seems like it's only called in > > the context of the class and outside imports shouldn't get it by default. Agreed. Will update and test python3. Thanks for the review! - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45487/#review126107 --- On March 30, 2016, 9:55 a.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45487/ > --- > > (Updated March 30, 2016, 9:55 a.m.) > > > Review request for Ambari and Jonathan Hurley. > > > Bugs: AMBARI-15631 > https://issues.apache.org/jira/browse/AMBARI-15631 > > > Repository: ambari > > > Description > --- > > The version_builder.py script should be a callable module > > > Diffs > - > > contrib/version-builder/example.py PRE-CREATION > contrib/version-builder/example.sh 7016997 > contrib/version-builder/version_builder.py 6c20a47 > > Diff: https://reviews.apache.org/r/45487/diff/ > > > Testing > --- > > Manual. No automated tests for contrib. > > > Thanks, > > Nate Cole > >
Re: Review Request 45540: Alerts Using the SKIPPED State Cause Stale Alert Notification
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45540/#review126304
---
Ship it!
Ship It!
- Nate Cole
On March 31, 2016, 9:33 a.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45540/
> ---
>
> (Updated March 31, 2016, 9:33 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate
> Cole.
>
>
> Bugs: AMBARI-15637
> https://issues.apache.org/jira/browse/AMBARI-15637
>
>
> Repository: ambari
>
>
> Description
> ---
>
> SCRIPT alerts which use the {{SKIPPED}} state as a way of preventing
> processing are causing instances of the Ambari Stale Alert to trigger. This
> is because the agents do not return these alerts in the heartbeat causing
> their timestamps not to update.
>
> STR:
> - Create a SCRIPT alert which returns {{SKIPPED}} as it's state.
> - Wait for the stale alert to trigger.
>
> {code}
> There are 1 stale alerts from 1 host(s): c6401.ambari.apache.org[NameNode
> Service RPC Queue Latency (Hourly) (2h 18m), NameNode Service RPC Processing
> Latency (Hourly) (2h 18m)]
> {code}
>
>
> Diffs
> -
>
> ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 92db07c
>
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
> f6aa1aa
> ambari-server/src/main/java/org/apache/ambari/server/state/AlertState.java
> 8cc245a
>
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java
> 775fe83
>
> Diff: https://reviews.apache.org/r/45540/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 45586: Provide an in-memory representation of available VDF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45586/ --- Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15652 https://issues.apache.org/jira/browse/AMBARI-15652 Repository: ambari Description --- * Load additional information out of hdp_urlinfo.json that will load VDF from a known location * Merging these separate VDF into one representation due to Ambari structures * Expose these available VDF via endpoint (included using established OperatingSystem and Repository sub-resources) * Allow creation of internal repo_version based on an available VDF Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/resources/VersionDefinitionResourceDefinition.java 67d9439 ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 0f09d94 ambari-server/src/main/java/org/apache/ambari/server/api/services/VersionDefinitionService.java e637850 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d1f8232 ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemRequest.java 16136db ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemResponse.java cc9bb70 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java 63f6b60 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperatingSystemResourceProvider.java b982583 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java a37364f ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 3ab5169 ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 0d87b68 ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 50bc30f ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 0ad24fe ambari-server/src/main/resources/version_definition.xsd 3c0399f ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java d729d2a ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 58f00e9 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java efdf84e ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java 707057f ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/hdp.json 37a6a60 ambari-server/src/test/resources/stacks/HDP/2.2.0/repos/version-2.2.0.4-123.xml PRE-CREATION contrib/version-builder/version_builder.py f209894 Diff: https://reviews.apache.org/r/45586/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
Review Request 45703: Allow skipping creation of repo files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45703/ --- Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Jonathan Hurley. Bugs: AMBARI-15697 https://issues.apache.org/jira/browse/AMBARI-15697 Repository: ambari Description --- RH Satellite requires that Ambari not manage the repositories for an OS. Add an attribute to OperatingSystems that gets persisted to do this work. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java f3197cb ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java a0b98f7 ambari-server/src/main/java/org/apache/ambari/server/controller/OperatingSystemResponse.java 06b4148 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java bb50820 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperatingSystemResourceProvider.java fd4a91f ambari-server/src/main/java/org/apache/ambari/server/orm/entities/OperatingSystemEntity.java 3c881a1 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java 6a36522 ambari-server/src/main/resources/custom_actions/scripts/install_packages.py c01dbb3 ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py 05751fa ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java 4da8896 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java e031fc8 ambari-server/src/test/python/custom_actions/TestInstallPackages.py ad4206b ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py aacd1f2 Diff: https://reviews.apache.org/r/45703/diff/ Testing --- Manual. Automated pending Thanks, Nate Cole
Re: Review Request 45712: Global Repeat Tolerance Value For Alerts
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45712/#review126966
---
Ship it!
Ship It!
- Nate Cole
On April 4, 2016, 6:47 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45712/
> ---
>
> (Updated April 4, 2016, 6:47 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush
> Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15702
> https://issues.apache.org/jira/browse/AMBARI-15702
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The global repeat tolerance value for all alert definitions is set on the
> {{cluster-env}} configuration. Unless an alert definition overrides this
> value, it will be used for any definition in the system. By default, this
> value will be set to 1, indicating that there is no retry tolerance and the
> alert state should be taken at face value.
>
> ```
> GET api/v1/clusters//configurations?type=cluster-env=
>
> "Config": {
> "cluster_name": "c1",
> "stack_id": "HDP-2.4"
> },
> "properties": {
> "command_retry_enabled": "true",
> "command_retry_max_time_in_sec": "600",
> ...
> "alerts_repeat_tolerance" : "1"
>...
> }
> ```
>
> The scope of work must also include
> - Default values if the property does not exist
> - Ambari upgrade work to ensure it's populated with a default value
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java
> 951b04b
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertResourceProvider.java
> 4c20c6c
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ClusterConfigChangedEvent.java
> dec2a33
>
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
> 2800ac6
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
> 38d05ab
>
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
> 77e36c8
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> 9e456eb
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
> b3241e0
>
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java
> f8a1f64
>
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
> df2ef46
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
> ea0547b
>
> Diff: https://reviews.apache.org/r/45712/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 46155: Re-Upgrade from 2.2 to 2.3+ Fails Due To Circular Symlink
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46155/#review128726
---
Ship it!
Ship It!
- Nate Cole
On April 13, 2016, 1:17 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46155/
> ---
>
> (Updated April 13, 2016, 1:17 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-15864
> https://issues.apache.org/jira/browse/AMBARI-15864
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> With Ambari 2.2.2.0-408 build, install 2.2.9.0 cluster (HA cluster)
> Perform EU to 2.4.2.0-178 till finalize step
> Downgrade back to 2.2.9
> Again run EU to 2.4.2.0
>
> The problem is that the downgrade unlinks configurations, getting rid of
> {{conf.backup}} and makes {{/etc//conf}} a directory again.
> However, the only code which bootstraps {{conf.backup}} is executed on
> installation of the stack. That action does not happen again between U->D->U.
>
> As a way to catch this scenario, {{conf_select.select()}} tries to be smart
> and checks for {{/etc//conf}} as a directory and if it finds that
> it is, it will create {{conf.backup}}.
>
> **The bug here is that the new {{/etc/<component {{/usr/hdp/current//conf}} which is a link back to
> {{/etc//conf}}. It should be linked to
> {{/etc//conf.backup}} instead.**
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
> b5de69d
>
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py
> 9f52b57
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py
> caf1430
>
> ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py
> 2bd9944
>
> Diff: https://reviews.apache.org/r/46155/diff/
>
>
> Testing
> ---
>
> OK
> --
> Total run:959
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 46159: VDF with newlines preventing valid repo files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46159/ --- Review request for Ambari, Alejandro Fernandez and Jonathan Hurley. Bugs: AMBARI-15867 https://issues.apache.org/jira/browse/AMBARI-15867 Repository: ambari Description --- When using a non-formatted VDF file with newlines in XML values, the newlines are being preserved. This prevents repo files from being generated correctly. (Also noticed a test failure that was easy to fix and a visual issue whereby non-manifest services like kerberos and AMS versions should be pulled from the stack) Diffs - ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java f855b05 ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java 4488f58 ambari-server/src/main/java/org/apache/ambari/server/state/repository/package-info.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/stack/TrimmingAdapter.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/stack/package-info.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 186ebe7 ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java d262df3 Diff: https://reviews.apache.org/r/46159/diff/ Testing --- Manual. Automated pending. Thanks, Nate Cole
