Re: Review Request 63354: Feature for supporting LDAP configuration from the UI
ain/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapConnectionConfigService.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/LdapConnectionTemplateFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/AttributeDetectorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/ChainedAttributeDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupMemberAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupNameAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupObjectClassDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/OccurrenceAndWeightBasedDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserGroupMemberAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserNameAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserObjectClassDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AmbariConfigurationDAO.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/DaoUtils.java cd3faf0 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AmbariConfigurationEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ConfigurationBaseEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java cd35c2c ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 015ec0a ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql eb9ca96 ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql dac3f28 ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql c321a38 ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8740ed7 ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 415589d ambari-server/src/main/resources/META-INF/persistence.xml e4045ef ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommandTest.java eaa4716 ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckOrderTest.java aa975e2 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProviderTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProviderTest.java ab60948 ambari-server/src/test/java/org/apache/ambari/server/ldap/LdapModuleFunctionalTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/domain/TestAmbariLdapConfigurationFactory.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/AmbariLdapFacadeTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapAttributeDetectionServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapConfigurationServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupMemberAttrDetectorTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/notifications/DispatchFactoryTest.java 382799c ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java 434a2a1 ambari-server/src/test/java/org/apache/ambari/server/orm/JdbcPropertyTest.java 427cede ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java 442414f ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java 4941bc7 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProviderTest.java 2362823 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/LdapServerPropertiesTest.java 5747408 Diff: https://reviews.apache.org/r/63354/diff/1/ Testing --- * unit tests written * manually: postman collection attached to the jira with sample calls Thanks, Laszlo Puskas
Re: Review Request 63354: Feature for supporting LDAP configuration from the UI
/LdapConnectionTemplateFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/AttributeDetectorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/ChainedAttributeDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupMemberAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupNameAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupObjectClassDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/OccurrenceAndWeightBasedDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserGroupMemberAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserNameAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserObjectClassDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AmbariConfigurationDAO.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/DaoUtils.java cd3faf0 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AmbariConfigurationEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ConfigurationBaseEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java cd35c2c ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 015ec0a ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql eb9ca96 ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql dac3f28 ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql c321a38 ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8740ed7 ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 415589d ambari-server/src/main/resources/META-INF/persistence.xml e4045ef ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommandTest.java eaa4716 ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckOrderTest.java aa975e2 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProviderTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProviderTest.java ab60948 ambari-server/src/test/java/org/apache/ambari/server/ldap/LdapModuleFunctionalTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/domain/TestAmbariLdapConfigurationFactory.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/AmbariLdapFacadeTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapAttributeDetectionServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapConfigurationServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupMemberAttrDetectorTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/notifications/DispatchFactoryTest.java 382799c ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java 434a2a1 ambari-server/src/test/java/org/apache/ambari/server/orm/JdbcPropertyTest.java 427cede ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java 442414f ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java 4941bc7 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProviderTest.java 2362823 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/LdapServerPropertiesTest.java 5747408 Diff: https://reviews.apache.org/r/63354/diff/1/ Testing --- * unit tests written * manually: postman collection attached to the jira with sample calls Thanks, Laszlo Puskas
Review Request 63354: Feature for supporting LDAP configuration from the UI
/main/java/org/apache/ambari/server/ldap/service/ads/detectors/ChainedAttributeDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupMemberAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupNameAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupObjectClassDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/OccurrenceAndWeightBasedDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserGroupMemberAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserNameAttrDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/detectors/UserObjectClassDetector.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AmbariConfigurationDAO.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/DaoUtils.java cd3faf0 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AmbariConfigurationEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ConfigurationBaseEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/security/authorization/RoleAuthorization.java cd35c2c ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 015ec0a ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql eb9ca96 ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql dac3f28 ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql c321a38 ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8740ed7 ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 415589d ambari-server/src/main/resources/META-INF/persistence.xml e4045ef ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommandTest.java eaa4716 ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckOrderTest.java aa975e2 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProviderTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProviderTest.java ab60948 ambari-server/src/test/java/org/apache/ambari/server/ldap/LdapModuleFunctionalTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/domain/TestAmbariLdapConfigurationFactory.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/AmbariLdapFacadeTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapAttributeDetectionServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapConfigurationServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/ldap/service/ads/detectors/GroupMemberAttrDetectorTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/notifications/DispatchFactoryTest.java 382799c ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java 434a2a1 ambari-server/src/test/java/org/apache/ambari/server/orm/JdbcPropertyTest.java 427cede ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java 442414f ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java 4941bc7 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProviderTest.java 2362823 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/LdapServerPropertiesTest.java 5747408 Diff: https://reviews.apache.org/r/63354/diff/1/ Testing --- * unit tests written * manually: postman collection attached to the jira with sample calls Thanks, Laszlo Puskas
Re: Review Request 62290: Validate kerberos.json files to ensure they meet the expected schema
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62290/#review185558 --- Ship it! Ship It! - Laszlo Puskas On Sept. 15, 2017, 2:34 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62290/ > --- > > (Updated Sept. 15, 2017, 2:34 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Laszlo Puskas, Mugdha Varadkar, and Sebastian > Toader. > > > Bugs: AMBARI-21938 > https://issues.apache.org/jira/browse/AMBARI-21938 > > > Repository: ambari > > > Description > --- > > Validate kerberos.json files to ensure they meet the expected schema. > > > Diffs > - > > ambari-server/pom.xml 29a820a329 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java > 0d1da9c743 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java > 5658133c9b > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java > 768a17e3b3 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java > 0eb3733584 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java > 911723b80b > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java > a17caadda0 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java > 74254e3fe3 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java > 5da33997ab > ambari-server/src/main/resources/stacks/HDP/2.6/kerberos_preconfigure.json > c9d8c91372 > > ambari-server/src/test/java/org/apache/ambari/server/stack/KerberosDescriptorTest.java > 0f1dd55044 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java > 09699c6fd6 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java > e891fdea4e > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java > 7fb56248e5 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java > 44812de55d > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java > d11962b9a3 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java > 8cdb39e02e > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java > b84223f421 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java > e4d3c90767 > ambari-server/src/test/resources/kerberos_descriptor_schema.json > PRE-CREATION > > > Diff: https://reviews.apache.org/r/62290/diff/3/ > > > Testing > --- > > Ensured the following unit tests passed: > > - org.apache.ambari.server.stack.KerberosDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosComponentDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosConfigurationDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosIdentityDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosKeytabDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosPrincipalDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosServiceDescriptorTest > > > Thanks, > > Robert Levas > >
Re: Review Request 62290: Validate kerberos.json files to ensure they meet the expected schema
> On Sept. 14, 2017, 12:40 p.m., Jonathan Hurley wrote: > > That's a lot of validation! I'm curious though - did you happen to look at > > something like http://json-schema.org/ instead of doing it explicitely in > > each descriptor class? +1 - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62290/#review185399 --- On Sept. 13, 2017, 7:52 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62290/ > --- > > (Updated Sept. 13, 2017, 7:52 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Laszlo Puskas, Mugdha Varadkar, and Sebastian > Toader. > > > Bugs: AMBARI-21938 > https://issues.apache.org/jira/browse/AMBARI-21938 > > > Repository: ambari > > > Description > --- > > Validate kerberos.json files to ensure they meet the expected schema. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java > 0d1da9c743 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java > 5658133c9b > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java > 768a17e3b3 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java > 7930e38f39 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java > 0eb3733584 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java > 911723b80b > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java > a17caadda0 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java > 74254e3fe3 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java > 5da33997ab > ambari-server/src/main/resources/stacks/HDP/2.6/kerberos_preconfigure.json > c9d8c91372 > > ambari-server/src/test/java/org/apache/ambari/server/stack/KerberosDescriptorTest.java > 0f1dd55044 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java > 09699c6fd6 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java > e891fdea4e > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java > 7fb56248e5 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java > d11962b9a3 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java > 8cdb39e02e > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java > b84223f421 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java > e4d3c90767 > > > Diff: https://reviews.apache.org/r/62290/diff/2/ > > > Testing > --- > > Ensured the following unit tests passed: > > - org.apache.ambari.server.stack.KerberosDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosComponentDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosConfigurationDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosIdentityDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosKeytabDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosPrincipalDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosServiceDescriptorTest > > > Thanks, > > Robert Levas > >
Re: Review Request 62290: Validate kerberos.json files to ensure they meet the expected schema
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62290/#review185405 --- Ship it! Ship It! - Laszlo Puskas On Sept. 13, 2017, 7:52 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62290/ > --- > > (Updated Sept. 13, 2017, 7:52 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Laszlo Puskas, Mugdha Varadkar, and Sebastian > Toader. > > > Bugs: AMBARI-21938 > https://issues.apache.org/jira/browse/AMBARI-21938 > > > Repository: ambari > > > Description > --- > > Validate kerberos.json files to ensure they meet the expected schema. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java > 0d1da9c743 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java > 5658133c9b > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java > 768a17e3b3 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java > 7930e38f39 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java > 0eb3733584 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java > 911723b80b > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java > a17caadda0 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java > 74254e3fe3 > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java > 5da33997ab > ambari-server/src/main/resources/stacks/HDP/2.6/kerberos_preconfigure.json > c9d8c91372 > > ambari-server/src/test/java/org/apache/ambari/server/stack/KerberosDescriptorTest.java > 0f1dd55044 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java > 09699c6fd6 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java > e891fdea4e > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java > 7fb56248e5 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java > d11962b9a3 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java > 8cdb39e02e > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java > b84223f421 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java > e4d3c90767 > > > Diff: https://reviews.apache.org/r/62290/diff/2/ > > > Testing > --- > > Ensured the following unit tests passed: > > - org.apache.ambari.server.stack.KerberosDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosComponentDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosConfigurationDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosIdentityDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosKeytabDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosPrincipalDescriptorTest > - org.apache.ambari.server.state.kerberos.KerberosServiceDescriptorTest > > > Thanks, > > Robert Levas > >
Re: Review Request 62064: Support for handling ldap admin group mappings
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62064/ --- (Updated Sept. 5, 2017, 9:26 a.m.) Review request for Ambari, Attila Doroszlai, Attila Magyar, and Robert Levas. Bugs: AMBARI-21873 https://issues.apache.org/jira/browse/AMBARI-21873 Repository: ambari Description --- This feature adds the possibility to handle users belonging to a defined LDAP groups as ambari administrators during the LDAP sync. The list of the groups that need to be considered is stored in the ambari property: ```java authorization.ldap.adminGroupMappingRules ``` The solution is to grant admin privileges to users belonging to these groups on LDPA sync. Warning: * changes in the LDAP group memberships will not be reflected in Ambari after the sync (eg.: administrator privileges won't be automatically revoked if users are removed from the groups listed in the property) * administrator privileges can be granted/removed by another administrator using the ambari UI, thus these actions can interfere * if groups are not synced, this property is not taken into account Diffs - ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java 7d8c683 Diff: https://reviews.apache.org/r/62064/diff/2/ Testing (updated) --- Manually: ambari-server sync-ldap —all - all users made admin ambari-server sync-ldap --users /tmp/users.csv - user imported, property not taken into account ambari-server sync-ldap —groups /tmp/groups.csv - the csv contains a group in the mapping rule, all users in LDAP belonging to the group imported and made admin ambari-server sync-ldap —groups /tmp/groups.csv - manually modified the admin (revoked admin provileges) - the ldap sync didn’t override the manual setting Uni tests OK Thanks, Laszlo Puskas
Re: Review Request 62064: Support for handling ldap admin group mappings
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62064/ --- (Updated Sept. 5, 2017, 8:30 a.m.) Review request for Ambari, Attila Doroszlai, Attila Magyar, and Robert Levas. Changes --- Fixed review issues. Bugs: AMBARI-21873 https://issues.apache.org/jira/browse/AMBARI-21873 Repository: ambari Description --- This feature adds the possibility to handle users belonging to a defined LDAP groups as ambari administrators during the LDAP sync. The list of the groups that need to be considered is stored in the ambari property: ```java authorization.ldap.adminGroupMappingRules ``` The solution is to grant admin privileges to users belonging to these groups on LDPA sync. Warning: * changes in the LDAP group memberships will not be reflected in Ambari after the sync (eg.: administrator privileges won't be automatically revoked if users are removed from the groups listed in the property) * administrator privileges can be granted/removed by another administrator using the ambari UI, thus these actions can interfere * if groups are not synced, this property is not taken into account Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java 7d8c683 Diff: https://reviews.apache.org/r/62064/diff/2/ Changes: https://reviews.apache.org/r/62064/diff/1-2/ Testing --- Manually: ambari-server sync-ldap —all - all users made admin ambari-server sync-ldap --users /tmp/users.csv - user imported, property not taken into account ambari-server sync-ldap —groups /tmp/groups.csv - the csv contains a group in the mapping rule, all users in LDAP belonging to the group imported and made admin ambari-server sync-ldap —groups /tmp/groups.csv - manually modified the admin (revoked admin provileges) - the ldap sync didn’t override the manual setting Unit tests ... in progress Thanks, Laszlo Puskas
Re: Review Request 62064: Support for handling ldap admin group mappings
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62064/ --- (Updated Sept. 4, 2017, 1:47 p.m.) Review request for Ambari, Attila Doroszlai, Attila Magyar, and Robert Levas. Bugs: AMBARI-21873 https://issues.apache.org/jira/browse/AMBARI-21873 Repository: ambari Description (updated) --- This feature adds the possibility to handle users belonging to a defined LDAP groups as ambari administrators during the LDAP sync. The list of the groups that need to be considered is stored in the ambari property: ```java authorization.ldap.adminGroupMappingRules ``` The solution is to grant admin privileges to users belonging to these groups on LDPA sync. Warning: * changes in the LDAP group memberships will not be reflected in Ambari after the sync (eg.: administrator privileges won't be automatically revoked if users are removed from the groups listed in the property) * administrator privileges can be granted/removed by another administrator using the ambari UI, thus these actions can interfere * if groups are not synced, this property is not taken into account Diffs - ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java 7d8c683 Diff: https://reviews.apache.org/r/62064/diff/1/ Testing --- Manually: ambari-server sync-ldap —all - all users made admin ambari-server sync-ldap --users /tmp/users.csv - user imported, property not taken into account ambari-server sync-ldap —groups /tmp/groups.csv - the csv contains a group in the mapping rule, all users in LDAP belonging to the group imported and made admin ambari-server sync-ldap —groups /tmp/groups.csv - manually modified the admin (revoked admin provileges) - the ldap sync didn’t override the manual setting Unit tests ... in progress Thanks, Laszlo Puskas
Review Request 62064: Support for handling ldap admin group mappings
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62064/ --- Review request for Ambari, Attila Doroszlai, Attila Magyar, and Robert Levas. Bugs: AMBARI-21873 https://issues.apache.org/jira/browse/AMBARI-21873 Repository: ambari Description --- This feature adds the possibility to handle users belonging to a defined LDAP groups as ambari administrators during the LDAP sync. The list of the groups that need to be considered is stored in the ambari property: ``` authorization.ldap.adminGroupMappingRules ``` The solution is to grant admin privileges to users belonging to these groups on LDPA sync. Warning: * changes in the LDAP group memberships will not be reflected in Ambari after the sync (eg.: administrator privileges won't be automatically revoked if users are removed from the groups listed in the property) * administrator privileges can be granted/removed by another administrator using the ambari UI, thus these actions can interfere * if groups are not synced, this property is not taken into account Diffs - ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java 7d8c683 Diff: https://reviews.apache.org/r/62064/diff/1/ Testing --- Manually: ambari-server sync-ldap —all - all users made admin ambari-server sync-ldap --users /tmp/users.csv - user imported, property not taken into account ambari-server sync-ldap —groups /tmp/groups.csv - the csv contains a group in the mapping rule, all users in LDAP belonging to the group imported and made admin ambari-server sync-ldap —groups /tmp/groups.csv - manually modified the admin (revoked admin provileges) - the ldap sync didn’t override the manual setting Unit tests ... in progress Thanks, Laszlo Puskas
Re: Review Request 61653: AMBARI-21723. Redundant type arguments (Java 8)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61653/#review182937 --- Ship it! Ship It! - Laszlo Puskas On Aug. 15, 2017, 10:23 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61653/ > --- > > (Updated Aug. 15, 2017, 10:23 a.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and > Sebastian Toader. > > > Bugs: AMBARI-21723 > https://issues.apache.org/jira/browse/AMBARI-21723 > > > Repository: ambari > > > Description > --- > > Removed type arguments that Java 8 can infer. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java > 91db7d02f694ec7d679b9c39ad4436eb90d02abc > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java > 3be66d42742735b6e3ff11a56796d0ab04e9084a > ambari-server/src/main/java/org/apache/ambari/server/agent/ActionQueue.java > 1e13e25c5572f8bef9e878c2159303d170ea8f47 > > ambari-server/src/main/java/org/apache/ambari/server/agent/AgentRequests.java > 8457bb86d4c6ddf28ab27558046e44742a90edda > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > 92ab3de56dd8def6c5ce4b265cd138ee7261abb7 > > ambari-server/src/main/java/org/apache/ambari/server/alerts/StaleAlertRunnable.java > 4f89eaa8cb1f1b51b368f05fd5ef349ada6a17b1 > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > 25fd14365eb5139710f689f05655d8a3e82baea8 > > ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java > b00391a44b2d576816346d73f865748da29e3f63 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ViewInstanceResourceDefinition.java > f7920a4249ffd5c5997bd416f5cbd155124cb39e > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > e991b720ddf37a1f2e29fafc74553b6ccfba77f4 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/parsers/JsonRequestBodyParser.java > 504bf04727e06e3e1d0597d0f4646b4467d8a366 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/persistence/PersistenceManagerImpl.java > 078dce88812b62cd4d51b94ba4120b54fa11bb5e > > ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java > 4cb220898004c5d33194db0ff72a6a277ca12396 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRunner.java > 90b95b3bd74e3e5dbbb1e77526659c7f9df651f8 > > ambari-server/src/main/java/org/apache/ambari/server/audit/AsyncAuditLogger.java > b843dd81cc0b01a1f9b905777167e560742efa89 > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java > 8f03655cd5d478d41b707bb212586e713ed0e84a > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RepositoryVersionEventCreator.java > d324b0464f9bde8f20272fe05c8a9dde73b386e0 > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java > 3042772ddca050ca0606288c621d0800fd5dc650 > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BootStrapImpl.java > bfe40ea8b67afbfc1ae8a81b15e898cdd210e9a2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java > 55356c78668a17c2a309d6821a072d29aa202267 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > b4d9608a8deee18c8b5f38cb12fc18b2ced0094c > > ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigGroupResponse.java > 937df46ffbb0ea184e41d68e2537cdfcf0a9b24c > > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java > f3c2ec871bf7e9250951472c9a7804d56939fc69 > > ambari-server/src/main/java/org/apache/ambari/server/controller/HostResponse.java > 765d25571d597a07809bc68e36027119f715a640 > > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java > 7b8f58db11b4c153daded199403f6836a877a9c5 > > ambari-server/src/main/java/org/apache/ambari/server/controller/LdapSyncRequest.java > c28e2221f84da45e17e5757e066e262d31fa1825 > > ambari-serve
Re: Review Request 61616: AMBARI-21714. Intermittent unit test failure in ExecutionCommandWrapperTest
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61616/#review182854 --- Ship it! Ship It! - Laszlo Puskas On Aug. 14, 2017, 11:09 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61616/ > --- > > (Updated Aug. 14, 2017, 11:09 a.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and > Sebastian Toader. > > > Bugs: AMBARI-21714 > https://issues.apache.org/jira/browse/AMBARI-21714 > > > Repository: ambari > > > Description > --- > > Move service addition to common setup. > > > Diffs > - > > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java > 52d54aa9e58d8bd3593095df3e87b70b3b2ffba8 > > > Diff: https://reviews.apache.org/r/61616/diff/1/ > > > Testing > --- > > All tests: > > ``` > Running org.apache.ambari.server.actionmanager.ExecutionCommandWrapperTest > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.814 sec - > in org.apache.ambari.server.actionmanager.ExecutionCommandWrapperTest > ``` > > Without `testGetExecutionCommand()`: > > ``` > Running org.apache.ambari.server.actionmanager.ExecutionCommandWrapperTest > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.456 sec - > in org.apache.ambari.server.actionmanager.ExecutionCommandWrapperTest > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 61024: Implement support for recommending LDAP configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61024/#review182388 --- Ship it! Ship It! - Laszlo Puskas On Aug. 8, 2017, 1:40 p.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61024/ > --- > > (Updated Aug. 8, 2017, 1:40 p.m.) > > > Review request for Ambari, Laszlo Puskas, Robert Levas, Sandor Magyari, and > Sebastian Toader. > > > Bugs: AMBARI-21545 > https://issues.apache.org/jira/browse/AMBARI-21545 > > > Repository: ambari > > > Description > --- > > Ldap configuration can be included in recommendation requests and validation > requests. Configuration is passed down to stack advisor in services.json > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationRequestSwagger.java > d6714f9 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationService.java > 0632361 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > 7ba1b18 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 356754d > > ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommandTest.java > eaa4716 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProviderTest.java > 6df8b8b > > > Diff: https://reviews.apache.org/r/61024/diff/2/ > > > Testing > --- > > - Tested the rest interface manually > - Wrote new unit tests > - Ambari-server unit tests: PENDING > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 61140: Ambari REST API to support setting rack info during add host
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61140/#review181555 --- Ship it! Ship It! - Laszlo Puskas On July 26, 2017, 3:42 p.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61140/ > --- > > (Updated July 26, 2017, 3:42 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Laszlo Puskas, > Robert Levas, and Sandor Magyari. > > > Bugs: AMBARI-21575 > https://issues.apache.org/jira/browse/AMBARI-21575 > > > Repository: ambari > > > Description > --- > > The ```POST > http://:/api/v1/clusters//hosts``` and > ```POST > http://:/api/v1/clusters//hosts/``` > REST API should accept rack info for the hosts being added to the cluster > similar to how updating hosts via ```PUT > http://:/api/v1/clusters//hosts``` is done. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java > 8ef42ba > > ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java > fe56760 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > fc42ff9 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java > a4cc6fc > > > Diff: https://reviews.apache.org/r/61140/diff/1/ > > > Testing > --- > > Manual testing the ```POST > http://:/api/v1/clusters//hosts``` and > ```POST > http://:/api/v1/clusters//hosts/``` > REST API. > > Unit tests: > Results : > > Tests run: 4815, Failures: 0, Errors: 0, Skipped: 35 > > [INFO] > [INFO] --- apache-rat-plugin:0.11:check (default) @ ambari-server --- > [INFO] RAT will not execute since it is configured to be skipped via system > property 'rat.skip'. > [INFO] > [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server --- > [INFO] skipping execute as per configuraion > [INFO] > [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server --- > [INFO] Starting audit... > Audit done. > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main SUCCESS [ 0.558 > s] > [INFO] Apache Ambari Project POM .. SUCCESS [ 0.003 > s] > [INFO] Ambari Views ... SUCCESS [ 2.423 > s] > [INFO] ambari-utility . SUCCESS [ 3.240 > s] > [INFO] ambari-metrics . SUCCESS [ 0.185 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 5.716 > s] > [INFO] Ambari Service Advisor . SUCCESS [ 0.382 > s] > [INFO] Ambari Server .. SUCCESS [51:41 > min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 51:54 min > [INFO] Finished at: 2017-07-26T16:38:04+02:00 > [INFO] Final Memory: 61M/511M > [INFO] > > > > Thanks, > > Sebastian Toader > >
Re: Review Request 60665: Add swagger annotations to the new AmbariConfigurationService class.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60665/#review179771 --- Ship it! Ship It! - Laszlo Puskas On July 6, 2017, 10:13 a.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60665/ > --- > > (Updated July 6, 2017, 10:13 a.m.) > > > Review request for Ambari, Attila Doroszlai, Laszlo Puskas, Oliver Szabo, and > Sebastian Toader. > > > Bugs: AMBARI-21307 > https://issues.apache.org/jira/browse/AMBARI-21307 > > > Repository: ambari > > > Description > --- > > Added swagger annotations to the new AmbariConfigurationService class. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationRequestSwagger.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationResponseSwagger.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationService.java > 0fa6e44 > > > Diff: https://reviews.apache.org/r/60665/diff/2/ > > > Testing > --- > > Manually checked the generated html. > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 60540: AMBARI-21339 logviewer started along with nimbus if supervisor is not running on the same machine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60540/ --- (Updated July 3, 2017, 9:53 a.m.) Review request for Ambari, Attila Doroszlai, Sumit Mohanty, and Sebastian Toader. Changes --- Fixed unit tests. Bugs: AMBARI-21339 https://issues.apache.org/jira/browse/AMBARI-21339 Repository: ambari Description --- Logviewer is only started along with the supervisor so inn cases when nimbus runs on a node not hosting supervisor logviever doesn't run. Solution: - in such cases the logviewr is started along with nimbus Diffs (updated) - ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus.py a6bc437 ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py 35f057c ambari-server/src/test/python/stacks/2.1/configs/default-storm-start.json 27cb63e ambari-server/src/test/python/stacks/2.1/configs/default.json e04e1eb ambari-server/src/test/python/stacks/2.1/configs/secured-storm-start.json 1b027b7 ambari-server/src/test/python/stacks/2.1/configs/secured.json 61b359c Diff: https://reviews.apache.org/r/60540/diff/2/ Changes: https://reviews.apache.org/r/60540/diff/1-2/ Testing --- Manually on dev env. - provision cluster with storm, nimbus and supervisor running on the same host -> logviewer started along with *supervisor* - provision cluster with storm, nimbus running "alone" on the node -> logviewer started along with *nimbus* Thanks, Laszlo Puskas
Review Request 60540: AMBARI-21339 logviewer started along with nimbus if supervisor is not running on the same machine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60540/ --- Review request for Ambari, Attila Doroszlai, Sumit Mohanty, and Sebastian Toader. Bugs: AMBARI-21339 https://issues.apache.org/jira/browse/AMBARI-21339 Repository: ambari Description --- Logviewer is only started along with the supervisor so inn cases when nimbus runs on a node not hosting supervisor logviever doesn't run. Solution: - in such cases the logviewr is started along with nimbus Diffs - ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus.py a6bc437 Diff: https://reviews.apache.org/r/60540/diff/1/ Testing --- Manually on dev env. - provision cluster with storm, nimbus and supervisor running on the same host -> logviewer started along with *supervisor* - provision cluster with storm, nimbus running "alone" on the node -> logviewer started along with *nimbus* Thanks, Laszlo Puskas
Re: Review Request 60269: AMBARI-21293. Allow skipping Swagger generation
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60269/#review178473
---
Ship it!
Ship It!
- Laszlo Puskas
On June 21, 2017, 9:24 a.m., Attila Doroszlai wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60269/
> ---
>
> (Updated June 21, 2017, 9:24 a.m.)
>
>
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, and Sebastian
> Toader.
>
>
> Bugs: AMBARI-21293
> https://issues.apache.org/jira/browse/AMBARI-21293
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Swagger generation can be skipped via `-Dswagger.skip` command-line
> parameter, similar to `-Dcheckstyle.skip` and `-Drat.skip`.
>
>
> Diffs
> -
>
> ambari-server/pom.xml 1ada974911cfe8ea205853f3d315e31ec0af0c66
>
>
> Diff: https://reviews.apache.org/r/60269/diff/1/
>
>
> Testing
> ---
>
> ```
> $ mvn -am -pl ambari-server clean test-compile
> ...
> [INFO] --- swagger-maven-plugin:3.1.4:generate (default) @ ambari-server ---
> [INFO] Reflections took 133 ms to scan 1 urls, producing 28 keys and 186
> values
> [INFO] Reading custom API reader:
> org.apache.ambari.swagger.AmbariSwaggerReader
> [INFO] Reflections took 92 ms to scan 1 urls, producing 28 keys and 186 values
> [INFO] This class exists both as top level and nested API:
> org.apache.ambari.server.api.services.HostService, treating it as top level
> API
> [INFO] Registering nested API: class
> org.apache.ambari.server.api.services.ServiceService
> [INFO] This class exists both as top level and nested API:
> org.apache.ambari.server.api.services.RequestService, treating it as top
> level API
> [INFO] Found 1 nested API's
> [INFO] Processing top level API: ViewPrivilegeService
> [INFO] Processing nested API: NestedApiRecord {nestedApi=class
> org.apache.ambari.server.api.services.ServiceService, parentApi=class
> org.apache.ambari.server.api.services.ClusterService,
> parentApiPath='/clusters', parentMethod=public
> org.apache.ambari.server.api.services.ServiceService
> org.apache.ambari.server.api.services.ClusterService.getServiceHandler(javax.ws.rs.core.Request,java.lang.String),
> parentMethodPath='{clusterName}/services'}
> [INFO] Will copy path params from parent method: [clusterName]
> [INFO] Processing top level API: ViewDataMigrationService
> [INFO] Processing top level API: ViewPermissionService
> [INFO] Processing top level API: ViewService
> [INFO] Processing top level API: GroupService
> [INFO] Processing top level API: UserPrivilegeService
> [INFO] Processing top level API: UserAuthorizationService
> [INFO] Processing top level API: GroupPrivilegeService
> [INFO] Processing top level API: ClusterService
> [INFO] Processing top level API: RootServiceService
> [INFO] Processing top level API: ViewVersionService
> [INFO] Processing top level API: HostService
> [INFO] Processing top level API: ActionService
> [INFO] Processing top level API: StacksService
> [INFO] Processing top level API: RequestService
> [INFO] Processing top level API: BlueprintService
> [INFO] Processing top level API: UserService
> [INFO] Processing top level API: SettingService
> [INFO] Processing top level API: ActiveWidgetLayoutService
> [INFO] Processing top level API: MemberService
> [INFO] Processing top level API: ViewInstanceService
> ...
> [INFO] BUILD SUCCESS
>
> $ mvn -am -pl ambari-server -Dswagger.skip clean test-compile
> ...
> [INFO] --- swagger-maven-plugin:3.1.4:generate (default) @ ambari-server ---
> [INFO] Swagger generation is skipped.
> ...
> [INFO] BUILD SUCCESS
> ```
>
>
> Thanks,
>
> Attila Doroszlai
>
>
Re: Review Request 60156: AMBARI-21158. Eliminate Maven warnings
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60156/#review178271 --- Ship it! Ship It! - Laszlo Puskas On June 16, 2017, 4:17 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60156/ > --- > > (Updated June 16, 2017, 4:17 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and > Sebastian Toader. > > > Bugs: AMBARI-21158 > https://issues.apache.org/jira/browse/AMBARI-21158 > > > Repository: ambari > > > Description > --- > > * specified versions where missing > * moved multiple `execution` occurrences under the same `executions` > * removed duplicates > > > Diffs > - > > ambari-funtest/pom.xml 3738106fcc48f2bb739dcfc741b93e96be970911 > ambari-metrics/ambari-metrics-host-monitoring/pom.xml > d6c1fab924a63de260840f6b208f1c18ccd148e1 > ambari-metrics/pom.xml b5859164f95fc6496fa78fd9cc005797223b140e > ambari-project/pom.xml 7e1b813957e7c154ee09b7ca57c490ced185dbcb > ambari-server/pom.xml 91c35e5b50f7db8340d83a0d3cba953b9174fd82 > ambari-web/pom.xml d7a1d186ccb95ca1eef11c59eae16c7836842285 > > > Diff: https://reviews.apache.org/r/60156/diff/1/ > > > Testing > --- > > ``` > mvn -DskipTests -fae clean package > ``` > > Only `python-client` fails, due to missing `setuptools` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 59956: Update Database Access Layer to Support New Database Schema for Improved User Account Management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59956/#review177984 --- Ship it! Ship It! - Laszlo Puskas On June 14, 2017, 9:22 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59956/ > --- > > (Updated June 14, 2017, 9:22 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader. > > > Bugs: AMBARI-21147 > https://issues.apache.org/jira/browse/AMBARI-21147 > > > Repository: ambari > > > Description > --- > > Update Database Access Layer to Support New Database Schema for Improved User > Account Management. > > * Update `org.apache.ambari.server.orm.entities.UserEntity` > * Update `org.apache.ambari.server.orm.dao.UserDAO` > * Add `org.apache.ambari.server.orm.entities.UserAuthenticationEntity` > * Add `org.apache.ambari.server.orm.dao.UserAuthenticationDAO` > > Note: Some changes will be revisited when updating the different > authentication processes to work with the improved user account management > code. > > > Diffs > - > > ambari-server/docs/api/generated/index.html 7ea4297b99 > ambari-server/docs/api/generated/swagger.json d7d54a510f > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > fb06e6d8a5 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 807bded873 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 8d262e269b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > aeba739a6d > > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java > f3c2ec871b > > ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java > 391213858e > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserRequest.java > 40818c8f48 > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserResponse.java > 5afacb70ef > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java > b35b2a8612 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProvider.java > 389f0b2bf2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java > 614f7abda1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java > c5c36e9942 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserAuthenticationDAO.java > PRE-CREATION > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserDAO.java > ce47c4c38c > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserAuthenticationEntity.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserEntity.java > 9011eaecec > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java > 195c55afa5 > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AuthenticationMethodNotAllowedException.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariAuthToLocalUserDetailsService.java > 1e4f6ead08 > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationProperties.java > 09422e51e3 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java > ce9a79023d > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java > b7ff297ce5 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthoritiesPopulator.java > d38d44c16f > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLocalUserProvider.java > 37d5d49c37 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java > 373552e6e1 > > ambari-server/src/m
Re: Review Request 59956: Update Database Access Layer to Support New Database Schema for Improved User Account Management
> On June 12, 2017, 2:06 p.m., Laszlo Puskas wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserEntity.java > > Lines 97 (patched) > > <https://reviews.apache.org/r/59956/diff/1/?file=1747000#file1747000line119> > > > > I think cascading should be removed here (AFAIK there were some issues > > where cascading was interfering with some DAO logic) > > Robert Levas wrote: > Cascade seems to be used a few lines up in this file... > ``` > @OneToMany(mappedBy = "user", cascade = CascadeType.ALL) > private Set memberEntities = new HashSet<>(); > ``` > > Other than that, a quick search in the source code reveals 72 occurances > of this, 42 instances are of `CascadeType.ALL`. > > So far my tests seem to work ok. > > Sebastian Toader wrote: > There was already an attempt to remove the Cascadings. I guess these are > left overs. > > Robert Levas wrote: > What is the alternative to cascading? If I remove a user, I will need to > add code in the DAO to first remove the user authentication records? > > Jonathan Hurley wrote: > I don't have problems with cascading as long as the entity relationships > are setup correctly for it. In many places, they are not and we need to rely > on the DAO to do the work. Maintaining these methods seems to be a source of > bugs, so if the entities here can be cascaded correctly, then I don't see why > it would be a problem. > > Robert Levas wrote: > The entity relationship is set up properly: > > ``` > CONSTRAINT FK_user_authentication_users FOREIGN KEY (user_id) REFERENCES > users(user_id) > ``` > > So how do we want to proceed? Laszlo? I am not against cascading; not using them may be a convention so that all operations are explicitly done in the code (its easier to follow what's going on, cascading induces some kind of "side effect"). But I am with Jonathan let's keep the cascading here. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59956/#review177610 --- On June 14, 2017, 9:22 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59956/ > ----------- > > (Updated June 14, 2017, 9:22 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader. > > > Bugs: AMBARI-21147 > https://issues.apache.org/jira/browse/AMBARI-21147 > > > Repository: ambari > > > Description > --- > > Update Database Access Layer to Support New Database Schema for Improved User > Account Management. > > * Update `org.apache.ambari.server.orm.entities.UserEntity` > * Update `org.apache.ambari.server.orm.dao.UserDAO` > * Add `org.apache.ambari.server.orm.entities.UserAuthenticationEntity` > * Add `org.apache.ambari.server.orm.dao.UserAuthenticationDAO` > > Note: Some changes will be revisited when updating the different > authentication processes to work with the improved user account management > code. > > > Diffs > - > > ambari-server/docs/api/generated/index.html 7ea4297b99 > ambari-server/docs/api/generated/swagger.json d7d54a510f > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > fb06e6d8a5 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 807bded873 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 8d262e269b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > aeba739a6d > > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java > f3c2ec871b > > ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java > 391213858e > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserRequest.java > 40818c8f48 > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserResponse.java > 5afacb70ef > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java > b35b2a8612 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActiveWi
Re: Review Request 60033: Ambari data purging
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60033/#review177883 --- Ship it! Ship It! - Laszlo Puskas On June 13, 2017, 4:43 a.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60033/ > --- > > (Updated June 13, 2017, 4:43 a.m.) > > > Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Laszlo > Puskas, Robert Levas, Sandor Magyari, and Vitalyi Brodetskyi. > > > Bugs: AMBARI-20749 > https://issues.apache.org/jira/browse/AMBARI-20749 > > > Repository: ambari > > > Description > --- > > Adding a couple of fixes to > https://issues.apache.org/jira/browse/AMBARI-20687 and renaming the > db-cleanup functionality to db-purge-history > > > Diffs > - > > ambari-server/sbin/ambari-server 7044096 > > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java > 70c3661 > > ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupDriver.java > 788290b > > ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupService.java > 880207c > > ambari-server/src/main/java/org/apache/ambari/server/cleanup/CleanupServiceImpl.java > 29a9041 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java > b9e1fab > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java > 38c0977 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostTaskDAO.java > eea8032 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyLogicalRequestDAO.java > 32a38da > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyLogicalTaskDAO.java > 3a72aed > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostTaskEntity.java > 37830b7 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyLogicalRequestEntity.java > 1536b80 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyLogicalTaskEntity.java > 2954863 > ambari-server/src/main/python/ambari-server.py 235ef95 > ambari-server/src/main/python/ambari_server/dbCleanup.py 6e16bc5 > ambari-server/src/main/python/ambari_server/setupActions.py 358bfc9 > > ambari-server/src/test/java/org/apache/ambari/server/cleanup/CleanupServiceImplTest.java > 7de5aae > > > Diff: https://reviews.apache.org/r/60033/diff/1/ > > > Testing > --- > > Manual testing: > ambari-server db-purge-history --cluster-name cl1 --from-date 2017-06-13 > > Verified that all request related records except the ones that belong to > upgrade were purged. > > > Unit tests: > > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main SUCCESS [ 0.811 > s] > [INFO] Apache Ambari Project POM .. SUCCESS [ 0.004 > s] > [INFO] Ambari Views ... SUCCESS [ 3.824 > s] > [INFO] utility SUCCESS [ 1.411 > s] > [INFO] ambari-metrics . SUCCESS [ 0.387 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 7.708 > s] > [INFO] Ambari Server .. SUCCESS [33:10 > min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 33:25 min > [INFO] Finished at: 2017-06-12T23:20:03+02:00 > [INFO] Final Memory: 56M/389M > [INFO] > > > > Thanks, > > Sebastian Toader > >
Re: Review Request 60081: Improve error message for Blueprints deployment with incorrect credential type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60081/#review177881 --- Ship it! Ship It! - Laszlo Puskas On June 14, 2017, 12:37 p.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60081/ > --- > > (Updated June 14, 2017, 12:37 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Laszlo Puskas, > Robert Levas, Robert Nettleton, Sandor Magyari, and Sebastian Toader. > > > Bugs: AMBARI-21242 > https://issues.apache.org/jira/browse/AMBARI-21242 > > > Repository: ambari > > > Description > --- > > Changed the exeption text to list valid credential types. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java > 2c95806 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java > e0735be > > > Diff: https://reviews.apache.org/r/60081/diff/1/ > > > Testing > --- > > - Tested the change manually > - Modified existing unit test for the API > - Run all unit tests in ambari-server, no errors. > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 59956: Update Database Access Layer to Support New Database Schema for Improved User Account Management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59956/#review177610 --- ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java Lines 277 (patched) <https://reviews.apache.org/r/59956/#comment251299> Is this method package private by design or should it be public? ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java Lines 308 (patched) <https://reviews.apache.org/r/59956/#comment251300> Does this method really need to be synchronized? (I understand this has only been moved here but maybe worths a short investigation) ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserEntity.java Lines 97 (patched) <https://reviews.apache.org/r/59956/#comment251301> I think cascading should be removed here (AFAIK there were some issues where cascading was interfering with some DAO logic) ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java Line 349 (original), 345 (patched) <https://reviews.apache.org/r/59956/#comment251302> Should this method be synchronized? ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java Lines 497 (patched) <https://reviews.apache.org/r/59956/#comment251303> Should this method be synchronized? ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java Line 483 (original), 524 (patched) <https://reviews.apache.org/r/59956/#comment251304> Should this method be synchronised? (The other methods need to be checked as well / i relize this might not relate to the problem this patch is to address) - Laszlo Puskas On June 9, 2017, 9:17 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59956/ > --- > > (Updated June 9, 2017, 9:17 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Laszlo Puskas, and Sebastian Toader. > > > Bugs: AMBARI-21147 > https://issues.apache.org/jira/browse/AMBARI-21147 > > > Repository: ambari > > > Description > --- > > Update Database Access Layer to Support New Database Schema for Improved User > Account Management. > > * Update `org.apache.ambari.server.orm.entities.UserEntity` > * Update `org.apache.ambari.server.orm.dao.UserDAO` > * Add `org.apache.ambari.server.orm.entities.UserAuthenticationEntity` > * Add `org.apache.ambari.server.orm.dao.UserAuthenticationDAO` > > Note: Some changes will be revisited when updating the different > authentication processes to work with the improved user account management > code. > > > Diffs > - > > ambari-server/docs/api/generated/index.html 7ea4297b99 > ambari-server/docs/api/generated/swagger.json d7d54a510f > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > fb06e6d8a5 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 807bded873 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 2a9d6c9ee0 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > 2f799b70c4 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java > 4fa236225d > > ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java > 391213858e > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserRequest.java > 40818c8f48 > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserResponse.java > 5afacb70ef > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java > b35b2a8612 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProvider.java > 389f0b2bf2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java > 614f7abda1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserResourceProvider.java > 2ec63dbaeb > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserAuthenticationDAO.java > PRE-CREATION > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserDAO.jav
Re: Review Request 59682: AMBARI-19149. Code cleanup: concatenation in debug messages, unnecessary toString calls
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59682/#review176436 --- Ship it! Ship It! - Laszlo Puskas On May 31, 2017, 11:07 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59682/ > --- > > (Updated May 31, 2017, 11:07 a.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and > Sebastian Toader. > > > Bugs: AMBARI-19149 > https://issues.apache.org/jira/browse/AMBARI-19149 > > > Repository: ambari > > > Description > --- > > * eliminate unnecessary `toString()` calls > * avoid concatenation and `String.format` in `debug()` and `trace()` calls > * replace few remaining commons-logging usages with SLF4J > * make sure `Logger LOG` is `final`, too > > > Diffs > - > > ambari-server/pom.xml c0aed7bbb381255cd2529d82baf34d677acbc94e > > ambari-server/src/main/java/org/apache/ambari/server/KdcServerConnectionVerification.java > 19b13aabdccfae0284950ff73cad6aa9b5ebe8ef > > ambari-server/src/main/java/org/apache/ambari/server/StateRecoveryManager.java > 826f2cabe313a2c9b297ee761dd01cd361c8d19f > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java > 982316dd3a24e1dca11935507b43cb630572f15f > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java > 3182b5070c1e852baf725fb79e0cdb1f11f4e158 > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java > e779f78e03a481fb5c59f2ceb9d8b77bea2bbed2 > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java > ff67eeaa290a1b209272b6e9e92c2b6a89d5fd66 > ambari-server/src/main/java/org/apache/ambari/server/agent/ActionQueue.java > 02f2dc8edb71898ce7032d262ac00c374d85c561 > > ambari-server/src/main/java/org/apache/ambari/server/agent/AgentRequests.java > 5ff15642a8b9ae0870df7aa6145b7d245e47e9a3 > > ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java > 419cae43c28a90e4209fee6789531213862642f8 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > a19025249476735b17a187ec0e6272363791a0bd > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java > 95f9b9ebb10475287ce2b95f69a8369604aca599 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java > 6649f8ac9e2751dfcee91112280d65c7a124915a > ambari-server/src/main/java/org/apache/ambari/server/agent/HostInfo.java > ce16f14a411705a8faee21f1ec6a532524e3d70e > ambari-server/src/main/java/org/apache/ambari/server/agent/Register.java > ec41a7bcc7ca23ab27a912c25db5a104a289e06f > > ambari-server/src/main/java/org/apache/ambari/server/agent/rest/AgentResource.java > 0690f7b1991047680b2d5d28711d3871c35980be > > ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java > 1387dedf063bb9f2d236a0752b62c39025344996 > > ambari-server/src/main/java/org/apache/ambari/server/api/rest/BootStrapResource.java > e5e1aa6f46a1c6d626abe842fd3b2f177a453754 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > 7e8d3cad60fc36f1af535349e11e3b78d92cfa95 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java > bf4d9307ee9c5ae8d9f22e0f670aa66e903202ed > > ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueService.java > 6a6b413a9d5719b68772b12205d2f9ea4612554b > > ambari-server/src/main/java/org/apache/ambari/server/api/services/parsers/JsonRequestBodyParser.java > b83534554cb81573ee9b87a3fdfd61dfff762ead > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java > 7731b222a6704217ae7f683de6de5becbc50f603 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > e4e2305dc9bbe83bd3773a8c273ef0b1c3827276 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/views/ViewDataMigrationService.java > 3b9c58d19b6ff234b2671d0919ec415edd00ecdf > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSHostStatusCollector.java > fa8a22afbf5bae89e158df46221c0094ee075619 > > ambari-server/src/main/java
Re: Review Request 59608: AMBARI-21133. Configure Ambari Identity fails with "Cannot run program: ambari-sudo.sh" on Ubuntu
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59608/#review176203 --- Ship it! Ship It! - Laszlo Puskas On May 26, 2017, 3:52 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59608/ > --- > > (Updated May 26, 2017, 3:52 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Laszlo > Puskas, Sandor Magyari, Sebastian Toader, and Vitalyi Brodetskyi. > > > Bugs: AMBARI-21133 > https://issues.apache.org/jira/browse/AMBARI-21133 > > > Repository: ambari > > > Description > --- > > `ambari-env.sh` not loaded due to `source` being undefined in `dash` and > potential syntax error in `ambari-env.sh` > > > Diffs > - > > ambari-server/conf/unix/ambari-env.sh > 33912b64c2afab9d3e4e606b9c5390f18fa3eabb > ambari-server/src/main/python/ambari_server_main.py > bf4f6d32d801ce0cfcd8e686a6ad54441649bc35 > > > Diff: https://reviews.apache.org/r/59608/diff/1/ > > > Testing > --- > > Manually tested on local Ubuntu 12 and 14 clusters. > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 59573: Exception message in validator generated based on ordered collections in order to be predictable.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59573/ --- (Updated May 25, 2017, 2:44 p.m.) Review request for Ambari, Attila Doroszlai, Jonathan Hurley, and Sandor Magyari. Bugs: AMBARI-21126 https://issues.apache.org/jira/browse/AMBARI-21126 Repository: ambari Description --- Assertions in the failing tests are mad on generated strings that were different due to the (unordered) collections used to generate them. Using sorted collections to make messages predictable. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java 759d9e9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java 8ead623 Diff: https://reviews.apache.org/r/59573/diff/2/ Testing (updated) --- Unit tests passed. Tests run: 4981, Failures: 0, Errors: 0, Skipped: 39 author lpuskas <lpus...@apache.org> Thu, 25 May 2017 11:52:59 +0200 (11:52 +0200) committer lpuskas <lpus...@apache.org> Thu, 25 May 2017 16:42:20 +0200 (16:42 +0200) commit 14acc0ae32a274184830a8e7bf847f5ede8ac953 Thanks, Laszlo Puskas
Re: Review Request 59573: Exception message in validator generated based on ordered collections in order to be predictable.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59573/ --- (Updated May 25, 2017, 2:39 p.m.) Review request for Ambari, Attila Doroszlai, Jonathan Hurley, and Sandor Magyari. Bugs: AMBARI-21126 https://issues.apache.org/jira/browse/AMBARI-21126 Repository: ambari Description --- Assertions in the failing tests are mad on generated strings that were different due to the (unordered) collections used to generate them. Using sorted collections to make messages predictable. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java 759d9e9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java 8ead623 Diff: https://reviews.apache.org/r/59573/diff/2/ Testing (updated) --- Unit tests passed. Tests run: 4981, Failures: 0, Errors: 0, Skipped: 39 Thanks, Laszlo Puskas
Re: Review Request 59573: Exception message in validator generated based on ordered collections in order to be predictable.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59573/ --- (Updated May 25, 2017, 1:23 p.m.) Review request for Ambari, Attila Doroszlai, Jonathan Hurley, and Sandor Magyari. Bugs: AMBARI-21126 https://issues.apache.org/jira/browse/AMBARI-21126 Repository: ambari Description --- Assertions in the failing tests are mad on generated strings that were different due to the (unordered) collections used to generate them. Using sorted collections to make messages predictable. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java 759d9e9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java 8ead623 Diff: https://reviews.apache.org/r/59573/diff/2/ Changes: https://reviews.apache.org/r/59573/diff/1-2/ Testing --- Unit tests running. (Both java 7 and 8) Thanks, Laszlo Puskas
Review Request 59573: Exception message in validator generated based on ordered collections in order to be predictable.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59573/ --- Review request for Ambari, Attila Doroszlai, Jonathan Hurley, and Sandor Magyari. Bugs: AMBARI-21126 https://issues.apache.org/jira/browse/AMBARI-21126 Repository: ambari Description --- Assertions in the failing tests are mad on generated strings that were different due to the (unordered) collections used to generate them. Using sorted collections to make messages predictable. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java 759d9e9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java 8ead623 Diff: https://reviews.apache.org/r/59573/diff/1/ Testing --- Unit tests running. (Both java 7 and 8) Thanks, Laszlo Puskas
Re: Review Request 59491: AMBARI-19149. Code cleanup: identical catch blocks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59491/#review175810 --- Ship it! Ship It! - Laszlo Puskas On May 23, 2017, 4:35 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59491/ > --- > > (Updated May 23, 2017, 4:35 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and > Sebastian Toader. > > > Bugs: AMBARI-19149 > https://issues.apache.org/jira/browse/AMBARI-19149 > > > Repository: ambari > > > Description > --- > > Utilize > [multi-catch](http://docs.oracle.com/javase/7/docs/technotes/guides/language/catch-multiple.html), > available since Java 7 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ServiceComponentHostEventWrapper.java > 83bee656cc1e4f9014bfe396eafe371e242bae26 > > ambari-server/src/main/java/org/apache/ambari/server/api/handlers/UpdateHandler.java > 7f6a484431c62277b6296b55075e2901489a633e > > ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java > 31cb0618de7743dc74378f75bbfeb3a8965b1994 > > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java > 7c40c8396cda807a6a01cda6b085d6b32f510b6e > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 9bc7f4a8686d128d0f51c764359cb8c41eff1b35 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > 4ad01a54a742db090a2cb9d7b7b04c688cd33fa4 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java > f8016a5f418e56c13bcd38ca1bdb3fadd7a2257f > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackDependencyResourceProvider.java > bc7492cc35221cad740a71fb55d14e5b329b42fd > > ambari-server/src/main/java/org/apache/ambari/server/orm/helpers/ScriptRunner.java > 1afd638d347e19d4fdb98696d7a6ffd29a9abd31 > > ambari-server/src/main/java/org/apache/ambari/server/scheduler/ExecutionScheduleManager.java > cab467fd4701589ab488b9f50315aa576483757e > > ambari-server/src/main/java/org/apache/ambari/server/security/encryption/AESEncryptor.java > 400cdafafba8bcb3c41efb76ac8e7aac40eda5d1 > > ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyServiceImpl.java > 1d6b73d9a573d53961d6be3f5639d2e0ebedd0ea > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java > cd191744311e7f84fa580ddae1ef77ca7d934c27 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/IPAKerberosOperationHandler.java > 5923b7249739c18c27077679ce0e2d9aa91b0271 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java > eea5071819627f5e94322ed7954851354d0693df > > ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java > ed25aea3d4f46b45df9698eb7655e60230377688 > > ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java > 14bc446a7ceb05a1d1eca2fb9a06b5b216ab38e9 > ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java > 850a40ba2af9d8d15f345af5c23e38018664dc13 > > ambari-server/src/main/java/org/apache/ambari/server/utils/ShellCommandUtil.java > a4408de6affc43a9dd7d9fe879a4f925e1cd5e22 > > ambari-server/src/test/java/org/apache/ambari/server/agent/LocalAgentSimulator.java > 892a1f571a1135fa83e5736310b83107b2b5612e > > > Diff: https://reviews.apache.org/r/59491/diff/1/ > > > Testing > --- > > ``` > Tests run: 4981, Failures: 0, Errors: 0, Skipped: 39 > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 59338: Backport Ambari-17694 - Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59338/#review175255 --- Ship it! Ship It! - Laszlo Puskas On May 17, 2017, 2:18 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59338/ > --- > > (Updated May 17, 2017, 2:18 p.m.) > > > Review request for Ambari, Anita Jebaraj, Laszlo Puskas, Sumit Mohanty, and > Sebastian Toader. > > > Bugs: AMBARI-21043 > https://issues.apache.org/jira/browse/AMBARI-21043 > > > Repository: ambari > > > Description > --- > > Backport Ambari-17694 to Ambari 2.5.1 because the patch was never committed > to branch-2.5 and thus did not make it into Ambari 2.5.0. > > Most of the changes from Ambari 2.4.x applied without much additional work, > however an upgrade path from Ambari 2.5.0 to Ambari 2.5.1 has been created > since users of Ambari 2.5.1 may have missed out on the needed changes. This > additional work was essentially copying > `org.apache.ambari.server.upgrade.UpgradeCatalog240#updateKAFKAConfigs` to > `org.apache.ambari.server.upgrade.UpgradeCatalog251#updateKAFKAConfigs` and > adding to the unit tests in > `org.apache.ambari.server.upgrade.UpgradeCatalog251Test`. > > See AMBARI-17694 for more information about the changes. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java > 77333b8bc4 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > cf0c37dd02 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog251.java > 146520fa11 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py > 1327090aa7 > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json > 60fa959f2d > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > d0f82af13d > > ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/kerberos.json > 501f969a93 > > ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java > 857047bd4d > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > 8baf496474 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog251Test.java > 4575998c50 > > > Diff: https://reviews.apache.org/r/59338/diff/1/ > > > Testing > --- > > Manually tested upgrades from Ambari 2.4.0 to Ambari 2.5.1, and from Ambari > 2.5.0 to Ambari 2.5.1 > > #Local test results: PENDING > > #Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 58990: Required properties for services listed in the blueprint are validated when all the configuration is available
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58990/ --- (Updated May 9, 2017, 9:38 a.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sandor Magyari. Bugs: AMBARI-20872 https://issues.apache.org/jira/browse/AMBARI-20872 Repository: ambari Description (updated) --- Validating required properties for services listed in the blueprint can't be accurate as further configurtion can be added in the cluster creation template. (Where the configuration is added varies by use cases and usage) This patch contains the change that defers this validation till all the configuration is together, namely till the cluster creation template is posted. The validation logic is the same as it was before. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 45a8c5c ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyValidator.java 146b424 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java 0e77301 ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 0608697 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58990/diff/2/ Testing (updated) --- Tested successfully on local environment. Added unit tests. Unit tests - success. Thanks, Laszlo Puskas
Re: Review Request 58990: Required properties for services listed in the blueprint are validated when all the configuration is available
> On May 4, 2017, 2:21 p.m., Robert Levas wrote: > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java > > Lines 38 (patched) > > <https://reviews.apache.org/r/58990/diff/1/?file=1708288#file1708288line38> > > > > Aren' there some password properities that are required when installing > > a service? If so, shouldn't the existance of the property be tested? Being > > that the fill set of configs is used when validating, I would expect that a > > required password property would have been set in the Cluster Creation > > Template. Password properties were ignored previously, that's why i left it this way. (I didn't want to change anything in the logic) - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58990/#review173901 ------- On May 9, 2017, 8:57 a.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58990/ > --- > > (Updated May 9, 2017, 8:57 a.m.) > > > Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, > and Sandor Magyari. > > > Bugs: AMBARI-20872 > https://issues.apache.org/jira/browse/AMBARI-20872 > > > Repository: ambari > > > Description > --- > > Validating required properties for services listed in the blueprint can't be > accurate as further configurtion can be added in the cluster creation > template. > (Where the configuration is added varies by use cases and usage) > > This patch contains the change that defers this validation till all the > configuration is together, namely till the cluster creation template is > posted. > The validation logic is the same as it was before (provided config entries > are checked against those marked as required in the stack) the way the > properties are collected has slightly changed: > > - before the properties were retrieved by traversing hostgroups > - in the current implementation required properties are retrieved by > traversing the services listed in the blueprint > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 45a8c5c > > ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyValidator.java > 146b424 > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java > 0e77301 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > 0608697 > > ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/58990/diff/2/ > > > Testing > --- > > Tested successfully on local environment. > Added unit tests. > > Unit tests running ... > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 58990: Required properties for services listed in the blueprint are validated when all the configuration is available
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58990/ --- (Updated May 9, 2017, 8:57 a.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sandor Magyari. Bugs: AMBARI-20872 https://issues.apache.org/jira/browse/AMBARI-20872 Repository: ambari Description --- Validating required properties for services listed in the blueprint can't be accurate as further configurtion can be added in the cluster creation template. (Where the configuration is added varies by use cases and usage) This patch contains the change that defers this validation till all the configuration is together, namely till the cluster creation template is posted. The validation logic is the same as it was before (provided config entries are checked against those marked as required in the stack) the way the properties are collected has slightly changed: - before the properties were retrieved by traversing hostgroups - in the current implementation required properties are retrieved by traversing the services listed in the blueprint Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 45a8c5c ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyValidator.java 146b424 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java 0e77301 ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 0608697 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58990/diff/2/ Changes: https://reviews.apache.org/r/58990/diff/1-2/ Testing --- Tested successfully on local environment. Added unit tests. Unit tests running ... Thanks, Laszlo Puskas
Re: Review Request 58991: AMBARI-20931. Wrong equality check in ValueAttributesInfo
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58991/#review173898 --- Ship it! Ship It! - Laszlo Puskas On May 4, 2017, 1:41 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58991/ > --- > > (Updated May 4, 2017, 1:41 p.m.) > > > Review request for Ambari, Laszlo Puskas, Sandor Magyari, and Vitalyi > Brodetskyi. > > > Bugs: AMBARI-20931 > https://issues.apache.org/jira/browse/AMBARI-20931 > > > Repository: ambari > > > Description > --- > > Fix typo in equals > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/ValueAttributesInfo.java > c41cd9ae621ba4887af45fe001806cee25d009b2 > > > Diff: https://reviews.apache.org/r/58991/diff/1/ > > > Testing > --- > > Ran relevant unit tests > > > Thanks, > > Attila Doroszlai > >
Review Request 58990: Required properties for services listed in the blueprint are validated when all the configuration is available
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58990/ --- Review request for Ambari, Attila Doroszlai, Robert Levas, and Robert Nettleton. Bugs: AMBARI-20872 https://issues.apache.org/jira/browse/AMBARI-20872 Repository: ambari Description --- Validating required properties for services listed in the blueprint can't be accurate as further configurtion can be added in the cluster creation template. (Where the configuration is added varies by use cases and usage) This patch contains the change that defers this validation till all the configuration is together, namely till the cluster creation template is posted. The validation logic is the same as it was before (provided config entries are checked against those marked as required in the stack) the way the properties are collected has slightly changed: - before the properties were retrieved by traversing hostgroups - in the current implementation required properties are retrieved by traversing the services listed in the blueprint Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 45a8c5c ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyValidator.java 146b424 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java 0e77301 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/RequiredConfigPropertiesValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58990/diff/1/ Testing --- Tested successfully on local environment. Added unit tests. Unit tests running ... Thanks, Laszlo Puskas
Re: Review Request 58735: The default value for sun.security.krb5.debug property is set to false.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/ --- (Updated April 27, 2017, 1:36 p.m.) Review request for Ambari, Robert Levas and Sandor Magyari. Bugs: AMBARI-20849 https://issues.apache.org/jira/browse/AMBARI-20849 Repository: ambari Description --- When enabling kerberos for a cluster containing KNOX the amount of loges generated is too high by default. Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml ad599e0 Diff: https://reviews.apache.org/r/58735/diff/1/ Testing --- Deployed a cluster locally. Checked the value of the property on the knox configuration page. Unit tests successfully ran. Thanks, Laszlo Puskas
Re: Review Request 58735: The default value for sun.security.krb5.debug property is set to false.
> On April 26, 2017, 6:36 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml > > Line 61 (original), 61 (patched) > > <https://reviews.apache.org/r/58735/diff/1/?file=1700384#file1700384line61> > > > > Does this value have to change for existing clusters? > > If so, should an Ambari Upgrade modify the property if unkerberized. > > Robert Levas wrote: > I don't think that we need to change this for existing clusters.. maybe > the users expect the logs to be there since it was turned on from the > beginning. > > If Kerberos is not enabled (or was disabled), then no logging will be > done... so the value of the property is irrelevant (when Kerberos is not > enabled). This value can also be changed from the UI. I don't think this change can be critical to existing clusters. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/#review173072 ------- On April 27, 2017, 1:15 p.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58735/ > --- > > (Updated April 27, 2017, 1:15 p.m.) > > > Review request for Ambari, Robert Levas and Sandor Magyari. > > > Bugs: AMBARI-20849 > https://issues.apache.org/jira/browse/AMBARI-20849 > > > Repository: ambari > > > Description > --- > > When enabling kerberos for a cluster containing KNOX the amount of loges > generated is too high by default. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml > ad599e0 > > > Diff: https://reviews.apache.org/r/58735/diff/1/ > > > Testing > --- > > Deployed a cluster locally. Checked the value of the property on the knox > configuration page. > Unit tests successfully ran. > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 58735: The default value for sun.security.krb5.debug property is set to false.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/ --- (Updated April 27, 2017, 1:15 p.m.) Review request for Ambari, Robert Levas and Sandor Magyari. Summary (updated) - The default value for sun.security.krb5.debug property is set to false. Bugs: AMBARI-20849 https://issues.apache.org/jira/browse/AMBARI-20849 Repository: ambari Description --- When enabling kerberos for a cluster containing KNOX the amount of loges generated is too high by default. Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml ad599e0 Diff: https://reviews.apache.org/r/58735/diff/1/ Testing --- Deployed a cluster locally. Checked the value of the property on the knox configuration page. Unit tests successfully ran. Thanks, Laszlo Puskas
Re: Review Request 58735: The default value for sun.security.krb5.debug property is set to fals.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/ --- (Updated April 27, 2017, 1:03 p.m.) Review request for Ambari, Robert Levas and Sandor Magyari. Bugs: AMBARI-20849 https://issues.apache.org/jira/browse/AMBARI-20849 Repository: ambari Description --- When enabling kerberos for a cluster containing KNOX the amount of loges generated is too high by default. Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml ad599e0 Diff: https://reviews.apache.org/r/58735/diff/1/ Testing (updated) --- Deployed a cluster locally. Checked the value of the property on the knox configuration page. Unit tests successfully ran. Thanks, Laszlo Puskas
Re: Review Request 58735: The default value for sun.security.krb5.debug property is set to fals.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/ --- (Updated April 27, 2017, 12:25 p.m.) Review request for Ambari, Robert Levas and Sandor Magyari. Bugs: AMBARI-20849 https://issues.apache.org/jira/browse/AMBARI-20849 Repository: ambari Description --- When enabling kerberos for a cluster containing KNOX the amount of loges generated is too high by default. Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml ad599e0 Diff: https://reviews.apache.org/r/58735/diff/1/ Testing (updated) --- Deployed a cluster locally. Checked the value of the property on the knox configuration page. Unit tests running ... Thanks, Laszlo Puskas
Re: Review Request 58735: The default value for sun.security.krb5.debug property is set to fals.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/ --- (Updated April 27, 2017, 12:22 p.m.) Review request for Ambari, Robert Levas and Sandor Magyari. Bugs: AMBARI-20849 https://issues.apache.org/jira/browse/AMBARI-20849 Repository: ambari Description --- When enabling kerberos for a cluster containing KNOX the amount of loges generated is too high by default. Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml ad599e0 Diff: https://reviews.apache.org/r/58735/diff/1/ Testing (updated) --- Deployed a cluster locally. Checked the value of the property on the knox configuration page. Thanks, Laszlo Puskas
Review Request 58735: The default value for sun.security.krb5.debug property is set to fals.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58735/ --- Review request for Ambari, Robert Levas and Sandor Magyari. Bugs: AMBARI-20849 https://issues.apache.org/jira/browse/AMBARI-20849 Repository: ambari Description --- When enabling kerberos for a cluster containing KNOX the amount of loges generated is too high by default. Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/gateway-site.xml ad599e0 Diff: https://reviews.apache.org/r/58735/diff/1/ Testing --- In progress. Thanks, Laszlo Puskas
Re: Review Request 58668: AMBARI-20832. Desired configs fetched in loop via HostResourceProvider.getHosts
> On April 24, 2017, 2:51 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java > > Line 681 (original), 681 (patched) > > <https://reviews.apache.org/r/58668/diff/1/?file=1698104#file1698104line681> > > > > Why not supply them anymore? Seems like a performance penality if you > > pass in `null` +1 - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58668/#review172788 --- On April 24, 2017, 2:43 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58668/ > --- > > (Updated April 24, 2017, 2:43 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Sandor Magyari, > and Sebastian Toader. > > > Bugs: AMBARI-20832 > https://issues.apache.org/jira/browse/AMBARI-20832 > > > Repository: ambari > > > Description > --- > > Fix trivial typo. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java > f604a7f8be9b2dad7810e46444ae79ead73f8a98 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java > ac6ac1d299947a3b4ba0f0f41f7a6f851b2a72db > > > Diff: https://reviews.apache.org/r/58668/diff/1/ > > > Testing > --- > > Unit tests: `HostResourceProviderTest` passed, other unit tests pending. > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 58565: AMBARI-20781. Hadoop QA test results link is broken
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58565/#review172494 --- Ship it! Ship It! - Laszlo Puskas On April 20, 2017, 2:53 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58565/ > --- > > (Updated April 20, 2017, 2:53 p.m.) > > > Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Sandor Magyari, Sumit > Mohanty, Sebastian Toader, and Yusaku Sako. > > > Bugs: AMBARI-20781 > https://issues.apache.org/jira/browse/AMBARI-20781 > > > Repository: ambari > > > Description > --- > > Link to Maven's output from test run. > Remove link to nonexistent page. > > > Diffs > - > > dev-support/test-patch.sh 934e09262e743b6acd42d2f5b440563a045dcc9d > > > Diff: https://reviews.apache.org/r/58565/diff/1/ > > > Testing > --- > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 58415: Topology configuration type validation prior to cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58415/ --- (Updated April 14, 2017, 7:29 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20755 https://issues.apache.org/jira/browse/AMBARI-20755 Repository: ambari Description --- Problem: Configuration types posted in the blueprint or in the cluster creation template need to be validated before the cluster provisioning is started and cluster resources are persisted in the database. This validation should be done when the topology configuration comes together so that it's not modified and enhanced further. Currently the topology configuration is modified after the validation thus on blueprint deployments the cluster provisioning may fail despite of the right configuration. Notes: - due to the nature of the topology validation, the validation logic and management of the validators have been extracted to a specialized service (before validators were attached to requests and the validation happened in the constructor) - some unit test cases became unecessary due to the above change, i removed them (and added other unit tests) - there are some formatting related hunks in the diff (i added them by mistake) The validators have been modified (as per internal discussions and test results) - configuration types are considered invalid if they are not in the stack ( regardless the services being deployed) - the hive database related validation checks for the mysql server in the list of components, not in the list of services Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java f24c138 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 1a14b01 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java b5d2f9d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java e37c68d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 37fb7d4 ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 2ac9950 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 392a53e ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyRequest.java cbc6642 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ChainedTopologyValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java 1351739 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java 591a124 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorService.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba4043 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java 73a80f6 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java 48d1351 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java a691cbc ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java 98ba592 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java fc7ac27 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 3ea17b4 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 4c88247 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 2d5978b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java 745b01b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58415/diff/3/ Testing (updated) --- Unit tests ran against trunk. (some unrelatet pyhon tests failed though) Manually tested on the local env. tested also the cherry-pick to branch-2.5. Thanks, Laszlo Puskas
Re: Review Request 58415: Topology configuration type validation prior to cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58415/ --- (Updated April 14, 2017, 2:08 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20755 https://issues.apache.org/jira/browse/AMBARI-20755 Repository: ambari Description (updated) --- Problem: Configuration types posted in the blueprint or in the cluster creation template need to be validated before the cluster provisioning is started and cluster resources are persisted in the database. This validation should be done when the topology configuration comes together so that it's not modified and enhanced further. Currently the topology configuration is modified after the validation thus on blueprint deployments the cluster provisioning may fail despite of the right configuration. Notes: - due to the nature of the topology validation, the validation logic and management of the validators have been extracted to a specialized service (before validators were attached to requests and the validation happened in the constructor) - some unit test cases became unecessary due to the above change, i removed them (and added other unit tests) - there are some formatting related hunks in the diff (i added them by mistake) The validators have been modified (as per internal discussions and test results) - configuration types are considered invalid if they are not in the stack ( regardless the services being deployed) - the hive database related validation checks for the mysql server in the list of components, not in the list of services Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java f24c138 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 1a14b01 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java b5d2f9d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java e37c68d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 37fb7d4 ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 2ac9950 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 392a53e ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyRequest.java cbc6642 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ChainedTopologyValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java 1351739 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java 591a124 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorService.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba4043 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java 73a80f6 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java 48d1351 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java a691cbc ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java 98ba592 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java fc7ac27 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 3ea17b4 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 4c88247 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 2d5978b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java 745b01b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58415/diff/3/ Testing (updated) --- Unit tests ran against trunk. (some unrelatet pyhon tests failed though) Manually tested on the local env. Testinfg on branch-2.5 Thanks, Laszlo Puskas
Re: Review Request 58415: Topology configuration type validation prior to cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58415/ --- (Updated April 14, 2017, 2:03 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20755 https://issues.apache.org/jira/browse/AMBARI-20755 Repository: ambari Description --- Problem: Configuration types posted in the blueprint or in the cluster creation template need to be validated before the cluster provisioning is started and cluster resources are persisted in the database. This validation should be done when the topology configuration comes together so that it's not modified and enhanced further. Currently the topology configuration is modified after the validation thus on blueprint deployments the cluster provisioning may fail despite of the right configuration. Notes: - due to the nature of the topology validation, the validation logic and management of the validators have been extracted to a specialized service (before validators were attached to requests and the validation happened in the constructor) - some unit test cases became unecessary due to the above change, i removed them (and added other unit tests) - there are some formatting related hunks in the diff (i added them by mistake) The validators have been modified (as per internal discussions and test results) - configuration types are considered invalid if they are not in the stack ( regardless the services being deployed) - the hive database related validation checks for the mysql server in the list of components, not in the list of services The patch is a draft, i am still massaging it (adding comments, cleaning etc...) Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java f24c138 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 1a14b01 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java b5d2f9d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java e37c68d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 37fb7d4 ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 2ac9950 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 392a53e ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyRequest.java cbc6642 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ChainedTopologyValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java 1351739 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java 591a124 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorService.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba4043 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java 73a80f6 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java 48d1351 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java a691cbc ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java 98ba592 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java fc7ac27 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 3ea17b4 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 4c88247 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 2d5978b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java 745b01b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58415/diff/3/ Changes: https://reviews.apache.org/r/58415/diff/2-3/ Testing --- Unit tests ran against trunk. (some unrelatet pyhon tests failed though) Manually tested on the local env. (Still need to do some testing) Thanks, Laszlo Puskas
Re: Review Request 58415: Topology configuration type validation prior to cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58415/ --- (Updated April 13, 2017, 3:33 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- Applied review notes. Bugs: AMBARI-20755 https://issues.apache.org/jira/browse/AMBARI-20755 Repository: ambari Description --- Problem: Configuration types posted in the blueprint or in the cluster creation template need to be validated before the cluster provisioning is started and cluster resources are persisted in the database. This validation should be done when the topology configuration comes together so that it's not modified and enhanced further. Currently the topology configuration is modified after the validation thus on blueprint deployments the cluster provisioning may fail despite of the right configuration. Notes: - due to the nature of the topology validation, the validation logic and management of the validators have been extracted to a specialized service (before validators were attached to requests and the validation happened in the constructor) - some unit test cases became unecessary due to the above change, i removed them (and added other unit tests) - there are some formatting related hunks in the diff (i added them by mistake) The validators have been modified (as per internal discussions and test results) - configuration types are considered invalid if they are not in the stack ( regardless the services being deployed) - the hive database related validation checks for the mysql server in the list of components, not in the list of services The patch is a draft, i am still massaging it (adding comments, cleaning etc...) Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java f24c138 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 1a14b01 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java b5d2f9d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java e37c68d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 37fb7d4 ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 2ac9950 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 392a53e ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyRequest.java cbc6642 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ChainedTopologyValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java 1351739 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java 591a124 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorService.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba4043 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java 73a80f6 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java 48d1351 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java a691cbc ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java 98ba592 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java fc7ac27 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 3ea17b4 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 4c88247 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 2d5978b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java 745b01b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58415/diff/2/ Changes: https://reviews.apache.org/r/58415/diff/1-2/ Testing --- Unit tests ran against trunk. (some unrelatet pyhon tests failed though) Manually tested on the local env. (Still need to do some testing) Thanks, Laszlo Puskas
Re: Review Request 58415: Topology configuration type validation prior to cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58415/ --- (Updated April 13, 2017, 2:13 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20755 https://issues.apache.org/jira/browse/AMBARI-20755 Repository: ambari Description --- Problem: Configuration types posted in the blueprint or in the cluster creation template need to be validated before the cluster provisioning is started and cluster resources are persisted in the database. This validation should be done when the topology configuration comes together so that it's not modified and enhanced further. Currently the topology configuration is modified after the validation thus on blueprint deployments the cluster provisioning may fail despite of the right configuration. Notes: - due to the nature of the topology validation, the validation logic and management of the validators have been extracted to a specialized service (before validators were attached to requests and the validation happened in the constructor) - some unit test cases became unecessary due to the above change, i removed them (and added other unit tests) - there are some formatting related hunks in the diff (i added them by mistake) The validators have been modified (as per internal discussions and test results) - configuration types are considered invalid if they are not in the stack ( regardless the services being deployed) - the hive database related validation checks for the mysql server in the list of components, not in the list of services The patch is a draft, i am still massaging it (adding comments, cleaning etc...) Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java f24c138 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 1a14b01 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java b5d2f9d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java e37c68d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 37fb7d4 ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 2ac9950 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 392a53e ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyRequest.java cbc6642 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ChainedTopologyValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java 1351739 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java 591a124 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorService.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba4043 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java 73a80f6 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java 48d1351 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java a691cbc ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java 98ba592 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java fc7ac27 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 3ea17b4 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 4c88247 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 2d5978b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java 745b01b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58415/diff/1/ Testing --- Unit tests ran against trunk. (some unrelatet pyhon tests failed though) Manually tested on the local env. (Still need to do some testing) Thanks, Laszlo Puskas
Review Request 58415: Topology configuration type validation prior to cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58415/ --- Review request for Ambari, Attila Doroszlai, Robert Levas, Sandor Magyari, and Sebastian Toader. Repository: ambari Description --- Problem: Configuration types posted in the blueprint or in the cluster creation template need to be validated before the cluster provisioning is started and cluster resources are persisted in the database. This validation should be done when the topology configuration comes together so that it's not modified and enhanced further. Currently the topology configuration is modified after the validation thus on blueprint deployments the cluster provisioning may fail despite of the right configuration. Notes: - due to the nature of the topology validation, the validation logic and management of the validators have been extracted to a specialized service (before validators were attached to requests and the validation happened in the constructor) - some unit test cases became unecessary due to the above change, i removed them (and added other unit tests) - there are some formatting related hunks in the diff (i added them by mistake) The validators have been modified (as per internal discussions and test results) - configuration types are considered invalid if they are not in the stack ( regardless the services being deployed) - the hive database related validation checks for the mysql server in the list of components, not in the list of services The patch is a draft, i am still massaging it (adding comments, cleaning etc...) Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java f24c138 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 1a14b01 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java b5d2f9d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java e37c68d ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 37fb7d4 ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 2ac9950 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 392a53e ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyRequest.java cbc6642 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ChainedTopologyValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java 1351739 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java 591a124 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/TopologyValidatorService.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba4043 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java 73a80f6 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java 48d1351 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java a691cbc ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java 98ba592 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java fc7ac27 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 3ea17b4 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 4c88247 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 2d5978b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java 745b01b ambari-server/src/test/java/org/apache/ambari/server/topology/validators/StackConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/58415/diff/1/ Testing --- Unit tests ran against trunk. (some unrelatet pyhon tests failed though) Manually tested on the local env. (Still need to do some testing) Thanks, Laszlo Puskas
Re: Review Request 58385: AMBARI-19149. Code cleanup: redundant modifiers (interfaces, enums)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58385/#review171717 --- Ship it! Ship It! - Laszlo Puskas On April 12, 2017, 11:39 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58385/ > --- > > (Updated April 12, 2017, 11:39 a.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-19149 > https://issues.apache.org/jira/browse/AMBARI-19149 > > > Repository: ambari > > > Description > --- > > * Members of interfaces are always `public`. > * Constructors of enums are always `private`. > * Member interfaces and enums are implicitly `static`. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java > a8b89a9e374ca35c7ae049aa5789d4c6a5f8bf1e > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java > b0550c01381e635ee90d58ab26030cb6fc89f56d > > ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java > 0d1ef13c211a35b4107eea7c8e4eaf69bfde7557 > > ambari-server/src/main/java/org/apache/ambari/server/alerts/AmbariPerformanceRunnable.java > 8faa2a1ae04d9d04d2d8a54fb0cd80f1e285c2f8 > > ambari-server/src/main/java/org/apache/ambari/server/api/handlers/RequestHandler.java > 9e2d92327ccd6888bace2a97d03bbbd5c9ba7981 > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/expressions/Expression.java > 206f2676cf57cb48b5ea0824c02a3fb17ed86106 > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/operators/LogicalOperator.java > 8881aadfc6555460fef33a3e7d056b3a4dc4549e > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/operators/Operator.java > f9c3acce332b490c199e9e8879821c3e547c8d5c > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/operators/RelationalOperator.java > cbcba0caaf27b1388e4429a5fa3c76162cd0018c > ambari-server/src/main/java/org/apache/ambari/server/api/query/Query.java > 8ac38ac29a992d3ccdb7fa3a1d4e4668e06893f5 > > ambari-server/src/main/java/org/apache/ambari/server/api/query/render/Renderer.java > b71d96fffed15d7b6a4f0746efa976656929d8b9 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstance.java > 3695807a767f2b56a5895ffa6769c99491660a29 > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactory.java > 0b5859b251fa0bbed35a68d239dafdf85eda1be6 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessor.java > ab25bb21bcc3c8f56dff7cd2c20634b1ab65d809 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultStatus.java > 6284879a02d8c73027e64eb62a1ee93287fc25d4 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/parsers/RequestBodyParser.java > dc16b3e2e7b378c4a89341a61feab0bb2226b16b > > ambari-server/src/main/java/org/apache/ambari/server/api/services/persistence/PersistenceManager.java > 9a8b677e99f7bd5f3eaee7a0e2fb8698fae48d9c > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > f5677c1a71c84f9caf96ec6d58bc6c9c73de6b8e > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommandType.java > 5aa1ea9d69871452b99c389af0cbc5c4c4d2c70e > ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java > 60d365ffc3e0f56272e52bfdf56f421caae9b30b > > ambari-server/src/main/java/org/apache/ambari/server/checks/UpgradeCheckGroup.java > 67cf4f1a805ee58863a2d0f82886feb5b7750e00 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > a8bfbf0ddc4160591a07d8ccca6172b8afc7adbb > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 91bfe09debd414fe63a365c15d8e94823495164d > > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java > d0005719a2b01c1c319782cd77b49a01b2454d5c > > ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java > f72f476d21466eaaae9dc1a6751490bdfc5ea2aa > > ambari-server/src/main/java/org/apache/ambari/server/controller/gsi
Re: Review Request 58083: AMBARI-20625. Hadoop QA javac warnings increase check is broken -- fix for ambari-server
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58083/#review170907
---
Ship it!
Ship It!
- Laszlo Puskas
On April 3, 2017, 5:49 p.m., Attila Doroszlai wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58083/
> ---
>
> (Updated April 3, 2017, 5:49 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Sandor
> Magyari, and Sebastian Toader.
>
>
> Bugs: AMBARI-20625
> https://issues.apache.org/jira/browse/AMBARI-20625
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Added `-Xlint:all` flag. Currently this has effect only on `ambari-server`
> module.
>
>
> Diffs
> -
>
> dev-support/test-patch.sh 661f871c6cce927688f17d1aa77aba37eae9bbe6
>
>
> Diff: https://reviews.apache.org/r/58083/diff/1/
>
>
> Testing
> ---
>
> Tested with patch that increased the number of warnings, got:
>
> ```
> {color:red}-1 javac{color}. The applied patch generated 496 javac
> compiler warnings (more than the trunk's current 494 warnings).
> ```
>
>
> Thanks,
>
> Attila Doroszlai
>
>
Re: Review Request 58153: AMBARI-19149. Code cleanup: String concatenation in StringBuilder.append()
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58153/#review170903 --- Ship it! Ship It! - Laszlo Puskas On April 3, 2017, 5:42 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58153/ > --- > > (Updated April 3, 2017, 5:42 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-19149 > https://issues.apache.org/jira/browse/AMBARI-19149 > > > Repository: ambari > > > Description > --- > > Replace string concatenation within `append()` calls with chained `append()`. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java > cca279b66eb4b5568169d49c66a95bb4d672de3a > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 537b9936e33cc9c2f714f8fe23f8232c0ca0d837 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ActionRequest.java > 6fdac2f32cc2044688961f80b7981d34f3d6e9f3 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ClusterResponse.java > b3a2def043df46efa16b6f4a9c7609c3e064a1c4 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ExecuteActionRequest.java > 351974a481d4b25a10283a6585a9d53e63755430 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceRequest.java > a8e6315f1ef44cd7719e65f31278254e85c1ba2a > > ambari-server/src/main/java/org/apache/ambari/server/controller/ShortTaskStatus.java > 975476f9d3e6d7e024546bdfc16450e95a0d36a1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/UserRequest.java > 282131a93b69ed8da8105298fc8264c64410eecb > > ambari-server/src/main/java/org/apache/ambari/server/customactions/ActionDefinitionManager.java > 7a2738b8cef9b43438df593c2af3fa89a952496e > > ambari-server/src/main/java/org/apache/ambari/server/orm/helpers/dbms/PostgresHelper.java > ad18428d043e026d7c90f30767d70576d87c2aa8 > > ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphNode.java > d2aac3c8e92e13252fd2aab6f9a3d7e75c2bba73 > ambari-server/src/main/java/org/apache/ambari/server/state/HostConfig.java > f95d843b78f46da2be2bf87f95740ecca3339aa2 > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java > 515cc63f740b4673c371573716cde9f3748dbb82 > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java > 9caff6a74554bf774cd2d2d38ac632a9812546a8 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > f994457f930b1e2a21e119f32f79d3b546ec09c8 > > ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/ganglia/GangliaPropertyProviderTest.java > 5ef71174146a681689538080fd463fae8bf066ae > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/RangerWebAlertConfigActionTest.java > 5d2e605339147b67c410725e41d52f1117403d74 > > > Diff: https://reviews.apache.org/r/58153/diff/1/ > > > Testing > --- > > ``` > $ mvn -am -pl ambari-server clean test > ... > Tests run: 4978, Failures: 0, Errors: 0, Skipped: 39 > ... > [INFO] BUILD SUCCESS > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 58147: AMBARI-19149. Code cleanup: StringBuffer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58147/#review170866 --- Ship it! Ship It! - Laszlo Puskas On April 3, 2017, 12:29 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58147/ > --- > > (Updated April 3, 2017, 12:29 p.m.) > > > Review request for Ambari, Laszlo Puskas, Sandor Magyari, and Sebastian > Toader. > > > Bugs: AMBARI-19149 > https://issues.apache.org/jira/browse/AMBARI-19149 > > > Repository: ambari > > > Description > --- > > Replace `StringBuffer` with `StringBuilder`. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/events/HostsRemovedEvent.java > 41458605ef21e00e65a5adbe2c4e5dace7ec54e3 > > ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialProvider.java > 0521aa1da4e85ec2688417951c01875012e41463 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java > 9f9ded61b383b4ee5d4e7740f9dadd15921d89fe > > > Diff: https://reviews.apache.org/r/58147/diff/1/ > > > Testing > --- > > ``` > $ mvn -am -pl ambari-server clean test-compile > ... > [INFO] BUILD SUCCESS > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 58082: AMBARI-20631. Hadoop QA link to RAT warnings is wrong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58082/#review170583 --- Ship it! Ship It! - Laszlo Puskas On March 30, 2017, 3:21 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58082/ > --- > > (Updated March 30, 2017, 3:21 p.m.) > > > Review request for Ambari, Laszlo Puskas, Sandor Magyari, and Sebastian > Toader. > > > Bugs: AMBARI-20631 > https://issues.apache.org/jira/browse/AMBARI-20631 > > > Repository: ambari > > > Description > --- > > * Replaced directory in URL with the one from correct link. > * Also fixed wrong ending color marker (which caused -1 to be shown in black > instead of red). > > > Diffs > - > > dev-support/test-patch.sh 661f871c6cce927688f17d1aa77aba37eae9bbe6 > > > Diff: https://reviews.apache.org/r/58082/diff/1/ > > > Testing > --- > > N/A, text-only change > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 58022: AMBARI-19149. Code cleanup: redundant type arguments
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58022/#review170377 --- Ship it! Ship It! - Laszlo Puskas On March 29, 2017, 11:06 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58022/ > --- > > (Updated March 29, 2017, 11:06 a.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-19149 > https://issues.apache.org/jira/browse/AMBARI-19149 > > > Repository: ambari > > > Description > --- > > Removed type arguments that Java 7 can infer. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/KdcServerConnectionVerification.java > 99e714dd5a6625825dc8cb3d8630f99da06bdce4 > ambari-server/src/main/java/org/apache/ambari/server/Role.java > 1fceb293ec8cecaa8a92657b3add8ab60254030e > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java > 14f02d29690dc08209814603cff1f0b838bdc247 > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java > 237214d83567458ecc18e063326c38db0e103f85 > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java > c8e45321ca40cacc011bda8818ee19350ecc88ba > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Request.java > 502c016f40f4b8a23a2bfc859a7eb71f71083f6b > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java > f7ceca21676ac4663a944acf8ba945b31cd67135 > ambari-server/src/main/java/org/apache/ambari/server/agent/ActionQueue.java > d6d5a35e2d97d7b4e721a613fd92b9a106aaf714 > > ambari-server/src/main/java/org/apache/ambari/server/agent/AgentRequests.java > d0b7cc6b2fed514db6e5b695b7904965181d5569 > > ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java > edf12514a2966d4f29877e93c702f107ed83f37a > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeat.java > 0323479b85f34ab002d77aec5cf1bb4d7690c8be > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > 7c833f6b5719c0393e274396826963c0a1f9e70c > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java > 02970a1dff2423d325ac8109cc0db2f1c5189ad9 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java > 0042f53c7edd17c8ab7c1b571839fd4945a3dd2e > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java > 4fbc161d9405e427b149d620208ac02e914e0ae8 > ambari-server/src/main/java/org/apache/ambari/server/agent/HostInfo.java > 988e78b5a903c9effeeb3c3fa80aaf9dce1e526f > > ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryReport.java > 40b2260a5c3e59bd2e7304576d71e8c0044c9c54 > > ambari-server/src/main/java/org/apache/ambari/server/agent/RegistrationResponse.java > c12ed82d3354d8b35e1bb053c62327ae40cc656d > > ambari-server/src/main/java/org/apache/ambari/server/agent/StatusCommand.java > 5dd9e6f7765a1cf9384c9295c5321398530a3857 > > ambari-server/src/main/java/org/apache/ambari/server/alerts/AmbariPerformanceRunnable.java > 11e87d0f759b80d096e112b818d2942eadffe49c > > ambari-server/src/main/java/org/apache/ambari/server/alerts/StaleAlertRunnable.java > 7cb679b60ec477bac50d14ec4d7d837adb2b8ef8 > > ambari-server/src/main/java/org/apache/ambari/server/api/AmbariCsrfProtectionFilter.java > 60c0e77ed5a40071eac32c331776c419ae97f12c > > ambari-server/src/main/java/org/apache/ambari/server/api/AmbariErrorHandler.java > 3f40ac883a0066f1fb59af1d8591edae7c9a494c > > ambari-server/src/main/java/org/apache/ambari/server/api/handlers/QueryCreateHandler.java > beb8ea26401b1651d616337d5e9e718214a563ff > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/QueryLexer.java > f76745424328b512e6341b8a70e0cdd68202da35 > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/QueryParser.java > 738681351d3fa73a7f2abd2cd76b49735ab69c34 > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/expressions/AbstractExpression.java > db4220ee593e8ef1bc0198112aa81453b91b416f > > ambari-server/src/main/java/org/apache/ambari/server/api/predicate/expressions/NotLog
Re: Review Request 57966: Validate HIVE dependency on MYSQL_SERVER on blueprint deployments based on the configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57966/ --- (Updated March 28, 2017, 2:26 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sebastian Toader. Bugs: AMBARI-20594 https://issues.apache.org/jira/browse/AMBARI-20594 Repository: ambari Description (updated) --- Problem: On blueprint deployments when the HIVE service is configured with default values the cluster provisioning fails if the MYSQL_SERVER component is not explicitly listed in the blueprint. The failure happens during the cluster provisioning and the root cause is hard to be detected. Solution: When the cluster template is posted (so that all the configuration is together) the cluster topology is validated: * if HIVE service is required * if the default hive database is to be used (internal mysql database) * if the MYSQL_SERVER is not listed in the blueprint then the cluster creation is interrupted and appropriate error is returned. pushed to *trunk*: author lpuskas <lpus...@apache.org> Mon, 27 Mar 2017 20:31:31 +0200 (20:31 +0200) committer lpuskas <lpus...@apache.org> Tue, 28 Mar 2017 16:23:40 +0200 (16:23 +0200) commit fa6aa0782e973ff975c7a6b69b410d7a3c5204d4 Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a63013a ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java d92193f ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57966/diff/5/ Testing --- Added unit tests. Unit tests successfully ran. Manually tested on local cluster: - blueprint with HIVE, default configuration, *no* MYSQL_SERVER listed - validation failed - blueprint with HIVE, default configuration, MYSQL_SERVER listed - validation passed - blueprint with HIVE, hive-env/hive_database set to custom value - validation passed Thanks, Laszlo Puskas
Re: Review Request 57993: AMBARI-20603. Hadoop QA compiles trunk twice
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57993/#review170307 --- Ship it! Ship It! - Laszlo Puskas On March 28, 2017, 12:15 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57993/ > --- > > (Updated March 28, 2017, 12:15 p.m.) > > > Review request for Ambari, Jaimin Jetly, Laszlo Puskas, Sandor Magyari, Sumit > Mohanty, Sebastian Toader, and Yusaku Sako. > > > Bugs: AMBARI-20603 > https://issues.apache.org/jira/browse/AMBARI-20603 > > > Repository: ambari > > > Description > --- > > * Avoid duplicate trunk compilation (caused by Hadoop-specific code) > * Move RAT check before javac warnings check > * Get rid of some Hadoop-specific code (not applicable to Ambari) > > > Diffs > - > > dev-support/test-patch.sh 222a79ab3d2582ef26cdadfd69133d3e099be16b > > > Diff: https://reviews.apache.org/r/57993/diff/1/ > > > Testing > --- > > Tested manually with 3 different "test" patches: > > * one that builds fine without problems > * another which introduces checkstyle violation > * one that introduces RAT violation > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 57966: Validate HIVE dependency on MYSQL_SERVER on blueprint deployments based on the configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57966/ --- (Updated March 28, 2017, 12:14 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sebastian Toader. Bugs: AMBARI-20594 https://issues.apache.org/jira/browse/AMBARI-20594 Repository: ambari Description --- Problem: On blueprint deployments when the HIVE service is configured with default values the cluster provisioning fails if the MYSQL_SERVER component is not explicitly listed in the blueprint. The failure happens during the cluster provisioning and the root cause is hard to be detected. Solution: When the cluster template is posted (so that all the configuration is together) the cluster topology is validated: * if HIVE service is required * if the default hive database is to be used (internal mysql database) * if the MYSQL_SERVER is not listed in the blueprint then the cluster creation is interrupted and appropriate error is returned. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a63013a ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java d92193f ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57966/diff/4/ Changes: https://reviews.apache.org/r/57966/diff/3-4/ Testing --- Added unit tests. Unit tests successfully ran. Manually tested on local cluster: - blueprint with HIVE, default configuration, *no* MYSQL_SERVER listed - validation failed - blueprint with HIVE, default configuration, MYSQL_SERVER listed - validation passed - blueprint with HIVE, hive-env/hive_database set to custom value - validation passed Thanks, Laszlo Puskas
Re: Review Request 57966: Validate HIVE dependency on MYSQL_SERVER on blueprint deployments based on the configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57966/ --- (Updated March 27, 2017, 8:12 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sebastian Toader. Bugs: AMBARI-20594 https://issues.apache.org/jira/browse/AMBARI-20594 Repository: ambari Description (updated) --- Problem: On blueprint deployments when the HIVE service is configured with default values the cluster provisioning fails if the MYSQL_SERVER component is not explicitly listed in the blueprint. The failure happens during the cluster provisioning and the root cause is hard to be detected. Solution: When the cluster template is posted (so that all the configuration is together) the cluster topology is validated: * if HIVE service is required * if the default hive database is to be used (internal mysql database) * if the MYSQL_SERVER is not listed in the blueprint then the cluster creation is interrupted and appropriate error is returned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a63013a ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java d92193f ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57966/diff/2/ Testing (updated) --- Added unit tests. Unit tests successfully ran. Manually tested on local cluster: - blueprint with HIVE, default configuration, *no* MYSQL_SERVER listed - validation failed - blueprint with HIVE, default configuration, MYSQL_SERVER listed - validation passed - blueprint with HIVE, hive-env/hive_database set to custom value - validation passed Thanks, Laszlo Puskas
Re: Review Request 57966: Validate HIVE dependency on MYSQL_SERVER on blueprint deployments based on the configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57966/ --- (Updated March 27, 2017, 7:45 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sebastian Toader. Changes --- Fixed failing unit tests. Formatted strings. Bugs: AMBARI-20594 https://issues.apache.org/jira/browse/AMBARI-20594 Repository: ambari Description --- Problem: On blueprint deployments when the HIVE service is configured with default values the cluster provisioning fails if the MYSQL_SERVER component is not explicitly listed in the blueprint. The filure happens during the cluster provisioning and the root cause is hard to be detected. Solution: When the cluster template is posted (so that all the configuration is together) the cluster topology is validated: * if HIVE service is required * if the default hive database is to be used (internal mysql database) * if the MYSQL_SERVER is not listed in the blueprint then the cluster creation is interrupted and appropriate error is returned. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a63013a ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java d92193f ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57966/diff/2/ Changes: https://reviews.apache.org/r/57966/diff/1-2/ Testing (updated) --- Added unit tests. Unit tests successfully ran. Manual testing in progress. Thanks, Laszlo Puskas
Re: Review Request 57966: Validate HIVE dependency on MYSQL_SERVER on blueprint deployments based on the configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57966/ --- (Updated March 27, 2017, 6:54 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sebastian Toader. Bugs: AMBARI-20594 https://issues.apache.org/jira/browse/AMBARI-20594 Repository: ambari Description (updated) --- Problem: On blueprint deployments when the HIVE service is configured with default values the cluster provisioning fails if the MYSQL_SERVER component is not explicitly listed in the blueprint. The filure happens during the cluster provisioning and the root cause is hard to be detected. Solution: When the cluster template is posted (so that all the configuration is together) the cluster topology is validated: * if HIVE service is required * if the default hive database is to be used (internal mysql database) * if the MYSQL_SERVER is not listed in the blueprint then the cluster creation is interrupted and appropriate error is returned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a63013a ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57966/diff/1/ Testing --- Added unit tests. Related unit tests being fixed. Manual testing in progress. Thanks, Laszlo Puskas
Review Request 57966: Validate HIVE dependency on MYSQL_SERVER on blueprint deployments based on the configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57966/ --- Review request for Ambari, Attila Doroszlai, Robert Levas, Robert Nettleton, and Sebastian Toader. Bugs: AMBARI-20594 https://issues.apache.org/jira/browse/AMBARI-20594 Repository: ambari Description --- Problem: On blueprint deployments when the HIVE service is configured with default values the cluster provisioning fails if the MYSQL_SERVER component is not explicitly listed in the blueprint. The filure happens during the cluster provisioning and the root cause is hard to be detected. Solution: When the cluster template is posted (so that all the configuration is together) the cluster topology is validated: * if hive service is required * if the default hive database is to be used * if the MYSQL_SERVER is not listed in the blueprint then the cluster creation is interrupted and appropriate error is returned. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java a63013a ambari-server/src/main/java/org/apache/ambari/server/topology/validators/HiveServiceValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/topology/validators/HiveServiceValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57966/diff/1/ Testing --- Added unit tests. Related unit tests being fixed. Manual testing in progress. Thanks, Laszlo Puskas
Re: Review Request 57874: Configuration types from both the blueprint and cluster template are validated before cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57874/ --- (Updated March 27, 2017, 11:07 a.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, and Sebastian Toader. Changes --- Updated / fixed unit tests, removed unnecessary code. Bugs: AMBARI-20542 https://issues.apache.org/jira/browse/AMBARI-20542 Repository: ambari Description --- Problem: When validating config types posted in the blueprint and cluster template need to be checked against config types for services in the blueprint available in the stack. Previously only config types in the cluster template were considered during the validation. Solution All the config types are taken into account during the above validation, by calling the appropriate internal api. Note: A bug has also been corrected in finding valid/invalid config types. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java 305c88e ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java bb10df9 Diff: https://reviews.apache.org/r/57874/diff/4/ Changes: https://reviews.apache.org/r/57874/diff/3-4/ Testing --- Manually on local env. Unit tests written. Thanks, Laszlo Puskas
Re: Review Request 57874: Configuration types from both the blueprint and cluster template are validated before cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57874/ --- (Updated March 23, 2017, 1:51 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, and Sebastian Toader. Changes --- Fixed review notes. Bugs: AMBARI-20542 https://issues.apache.org/jira/browse/AMBARI-20542 Repository: ambari Description --- Problem: When validating config types posted in the blueprint and cluster template need to be checked against config types for services in the blueprint available in the stack. Previously only config types in the cluster template were considered during the validation. Solution All the config types are taken into account during the above validation, by calling the appropriate internal api. Note: A bug has also been corrected in finding valid/invalid config types. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java 18d08b9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java 24fa8b8 Diff: https://reviews.apache.org/r/57874/diff/2/ Changes: https://reviews.apache.org/r/57874/diff/1-2/ Testing --- Manually on local env. Unit tests written. Thanks, Laszlo Puskas
Re: Review Request 57874: Configuration types from both the blueprint and cluster template are validated before cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57874/ --- (Updated March 23, 2017, 1:54 p.m.) Review request for Ambari, Attila Doroszlai, Robert Levas, and Sebastian Toader. Bugs: AMBARI-20542 https://issues.apache.org/jira/browse/AMBARI-20542 Repository: ambari Description --- Problem: When validating config types posted in the blueprint and cluster template need to be checked against config types for services in the blueprint available in the stack. Previously only config types in the cluster template were considered during the validation. Solution All the config types are taken into account during the above validation, by calling the appropriate internal api. Note: A bug has also been corrected in finding valid/invalid config types. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java 18d08b9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java 24fa8b8 Diff: https://reviews.apache.org/r/57874/diff/3/ Changes: https://reviews.apache.org/r/57874/diff/2-3/ Testing --- Manually on local env. Unit tests written. Thanks, Laszlo Puskas
Review Request 57874: Configuration types from both the blueprint and cluster template are validated before cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57874/ --- Review request for Ambari, Attila Doroszlai, Robert Levas, and Sebastian Toader. Bugs: AMBARI-20542 https://issues.apache.org/jira/browse/AMBARI-20542 Repository: ambari Description --- Problem: When validating config types posted in the blueprint and cluster template need to be checked against config types for services in the blueprint available in the stack. Previously only config types in the cluster template were considered during the validation. Solution All the config types are taken into account during the above validation, by calling the appropriate internal api. Note: A bug has also been corrected in finding valid/invalid config types. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java 18d08b9 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java 24fa8b8 Diff: https://reviews.apache.org/r/57874/diff/1/ Testing --- Manually on local env. Unit tests written. Thanks, Laszlo Puskas
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 20, 2017, 11:48 a.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- Fixed import order. Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Note: Created new package for topology validators and moved existing validators into it. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/6/ Changes: https://reviews.apache.org/r/57733/diff/5-6/ Testing --- Manually tested on local environment. Unit tests succeeded. Thanks, Laszlo Puskas
Re: Review Request 57770: Disabling security fails with AttributeError
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57770/#review169414
---
Ship it!
Ship It!
- Laszlo Puskas
On March 20, 2017, 9:07 a.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57770/
> ---
>
> (Updated March 20, 2017, 9:07 a.m.)
>
>
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene
> Chekanskiy, Laszlo Puskas, and Sebastian Toader.
>
>
> Bugs: AMBARI-20494
> https://issues.apache.org/jira/browse/AMBARI-20494
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Disabling security is failing with :
> ```
> Stack Advisor reported an error: AttributeError: 'NoneType' object has no
> attribute 'replace'
> StdOut file: /var/run/ambari-server/stack-recommendations/26/stackadvisor.out
>
> StdErr file: /var/run/ambari-server/stack-recommendations/26/stackadvisor.err
> ```
>
> Error file shows :
> ```
> Traceback (most recent call last):
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 166,
> in
> main(sys.argv)
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 116,
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
> File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py",
> line 775, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
> File
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.5/services/stack_advisor.py",
> line 480, in recommendStormConfigurations
> storm_nimbus_impersonation_acl.replace('{{storm_bare_jaas_principal}}',
> storm_bare_jaas_principal)
> AttributeError: 'NoneType' object has no attribute 'replace'
> ```
>
>
> Diffs
> -
>
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
> 13151eb
> ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py
> 693b067
>
>
> Diff: https://reviews.apache.org/r/57770/diff/1/
>
>
> Testing
> ---
>
> Manually tested
>
> # Local test results:
> ```
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 1:15.026s
> [INFO] Finished at: Sat Mar 18 21:16:02 EDT 2017
> [INFO] Final Memory: 71M/1372M
> [INFO]
>
> ```
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 20, 2017, 10:38 a.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- - Added null checks - added test for equals - validator list extracted into a member Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Note: Created new package for topology validators and moved existing validators into it. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/5/ Changes: https://reviews.apache.org/r/57733/diff/4-5/ Testing --- Manually tested on local environment. Unit tests succeeded. Thanks, Laszlo Puskas
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
> On March 20, 2017, 8:54 a.m., Sebastian Toader wrote: > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java > > Lines 30 (patched) > > <https://reviews.apache.org/r/57733/diff/4/?file=1667947#file1667947line30> > > > > Implement equals and hashCode such as instances of this validator can > > be stored in hashmaps, sets etc. As the class doesn't have any members the default (super) implementation is enough here. > On March 20, 2017, 8:54 a.m., Sebastian Toader wrote: > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java > > Lines 37 (patched) > > <https://reviews.apache.org/r/57733/diff/4/?file=1667947#file1667947line37> > > > > Could ```topology.getConfiguration()``` ever be null? This is always set to a value, i can add a null check however. > On March 20, 2017, 8:54 a.m., Sebastian Toader wrote: > > ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java > > Lines 37 (patched) > > <https://reviews.apache.org/r/57733/diff/4/?file=1667951#file1667951line37> > > > > You could use ```EqualsVerifier``` to ensure that the ```equals``` and > > ```hashCode``` methods are properly implemented and are not broken > > inadvertently in the future. Added a simple check. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/#review169394 --- On March 18, 2017, 9:59 a.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57733/ > --- > > (Updated March 18, 2017, 9:59 a.m.) > > > Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. > > > Bugs: AMBARI-20488 > https://issues.apache.org/jira/browse/AMBARI-20488 > > > Repository: ambari > > > Description > --- > > When a cluster is created using blueprints it is possible to pass > configuration in the cluster creation template (that is posted after the > blueprint isent in) > On cluster provisioning configurations are validated and if errors are found > the cluster provisioning is interrupted. As the configuration validation is > done after the cluster resources are persisted it's not possible to resend > the (possibly corrected) cluster template and restart the cluster > provisioning. > Solution: > Perform the configuration type validation before the cluster resources are > persisted. > > Note: > Created new package for topology validators and moved existing validators > into it. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java > 2c8d09a > > ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java > 41812c3 > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java > c64da62 > > ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java > 8ab3a43 > > ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/57733/diff/4/ > > > Testing > --- > > Manually tested on local environment. > Unit tests succeeded. > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 18, 2017, 9:59 a.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Note: Created new package for topology validators and moved existing validators into it. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/4/ Changes: https://reviews.apache.org/r/57733/diff/3-4/ Testing --- Manually tested on local environment. Unit tests succeeded. Thanks, Laszlo Puskas
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 17, 2017, 5:45 p.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Note: Created new package for topology validators and moved existing validators into it. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/3/ Testing (updated) --- Manually tested on local environment. Unit tests succeeded. Thanks, Laszlo Puskas
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 17, 2017, 5:45 p.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- Fixed unit tests. Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Note: Created new package for topology validators and moved existing validators into it. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/3/ Changes: https://reviews.apache.org/r/57733/diff/2-3/ Testing --- Manually tested on local environment. Unit tests running. Thanks, Laszlo Puskas
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 17, 2017, 5:10 p.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description (updated) --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Note: Created new package for topology validators and moved existing validators into it. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/2/ Testing --- Manually tested on local environment. Unit tests running. Thanks, Laszlo Puskas
Re: Review Request 57733: Configuration types are validated before the cluster resources are persisted.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57733/ --- (Updated March 17, 2017, 5:07 p.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20488 https://issues.apache.org/jira/browse/AMBARI-20488 Repository: ambari Description --- When a cluster is created using blueprints it is possible to pass configuration in the cluster creation template (that is posted after the blueprint isent in) On cluster provisioning configurations are validated and if errors are found the cluster provisioning is interrupted. As the configuration validation is done after the cluster resources are persisted it's not possible to resend the (possibly corrected) cluster template and restart the cluster provisioning. Solution: Perform the configuration type validation before the cluster resources are persisted. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 2c8d09a ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 41812c3 ambari-server/src/main/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidator.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/validators/RequiredPasswordValidator.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequestTest.java c64da62 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java 8ab3a43 ambari-server/src/test/java/org/apache/ambari/server/topology/validators/ClusterConfigTypeValidatorTest.java PRE-CREATION Diff: https://reviews.apache.org/r/57733/diff/2/ Changes: https://reviews.apache.org/r/57733/diff/1-2/ Testing --- Manually tested on local environment. Unit tests running. Thanks, Laszlo Puskas
Re: Review Request 57544: Atlas MetaData server start fails while granting permissions to HBase tables after unkerberizing the cluster
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57544/#review168895 --- Ship it! Ship It! - Laszlo Puskas On March 12, 2017, 3:37 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57544/ > --- > > (Updated March 12, 2017, 3:37 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Laszlo Puskas, and Sebastian Toader. > > > Bugs: AMBARI-20408 > https://issues.apache.org/jira/browse/AMBARI-20408 > > > Repository: ambari > > > Description > --- > > STR > 1. Deploy HDP-2.5.0.0 with Ambari-2.5.0.0 (secure MIT cluster installed via > blueprint) > 2. Express Upgrade the cluster to 2.6.0.0 > 3. Disable Kerberos > 4. Observed that Atlas Metadata server start failed with below errors: > > ``` > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py", > line 249, in > MetadataServer().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 282, in execute > method(env) > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 720, in restart > self.start(env, upgrade_type=upgrade_type) > File > "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py", > line 102, in start > user=params.hbase_user > File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", > line 155, in __init__ > self.env.run() > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 160, in run > self.run_action(resource, action) > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 124, in run_action > provider_action() > File > "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", > line 262, in action_run > tries=self.resource.tries, try_sleep=self.resource.try_sleep) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 72, in inner > result = function(command, **kwargs) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 102, in checked_call > tries=tries, try_sleep=try_sleep, > timeout_kill_strategy=timeout_kill_strategy) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 150, in _call_wrapper > result = _call(command, **kwargs_copy) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 303, in _call > raise ExecutionFailed(err_msg, code, out, err) > resource_management.core.exceptions.ExecutionFailed: Execution of 'cat > /var/lib/ambari-agent/tmp/atlas_hbase_setup.rb | hbase shell -n' returned 1. > Hortonworks # > This is MOTD message, added for testing in qe infra > atlas_titan > ATLAS_ENTITY_AUDIT_EVENTS > atlas > TABLE > ATLAS_ENTITY_AUDIT_EVENTS > atlas_titan > 2 row(s) in 0.2000 seconds > > nil > TABLE > ATLAS_ENTITY_AUDIT_EVENTS > atlas_titan > 2 row(s) in 0.0030 seconds > > nil > java exception > ERROR Java::OrgApacheHadoopHbaseIpc::RemoteWithExtrasException: > org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered > coprocessor service found for name AccessControlService in region > hbase:acl,,1480905643891.19e697cf0c4be8a99c54e39aea069b29. > at > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7692) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1897) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1879) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32299) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.jav
Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167985
---
Ship it!
Ship It!
- Laszlo Puskas
On March 5, 2017, 9:38 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
>
> (Updated March 5, 2017, 9:38 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader,
> and Sid Wagle.
>
>
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
>
>
> Repository: ambari
>
>
> Description
> ---
>
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
>
> ```
> Execution of '/usr/bin/kinit -c
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
> -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab >
> /dev/null' returned 1. kinit: Client not found in Kerberos database while
> getting initial credentials
> ```
>
> This issue is also seen in /var/log/krb5kdc.log:
>
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND:
> /etc/security/keytabs/spnego.service.key...@example.com for
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
>
> #Cause
> It appears that the HBASE alerts.json file
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the
> `kerberos_keytab` and `kerberos_principal` properties.
>
> ```
> {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization
> of the HBase Master exceeds certain warning and critical thresholds. It
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
> "type": "METRIC",
> "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> },
> "reporting": {
> "ok": {
> "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
> "text": "{1} CPU, load {0:.1%}",
> "value": 200
> },
> "critical": {
> "text": "{1} CPU, load {0:.1%}",
> "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
> },
> "jmx": {
> "property_list": [
> "java.lang:type=OperatingSystem/SystemCpuLoad",
> "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
> }
> }
> }
> ```
>
> Notice:
> ```
> "kerberos_keytab":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
>
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
>
> ```
> "kerberos_principal":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> 2a684dc
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json
> 1b3ae25
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
> 7ee66ef
>
>
> Diff: https://reviews.apache.org/r/57324/diff/1/
>
>
> Testing
> ---
>
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari
> 2.4.2 to Ambari 2.5.0
>
> # Local test results: PASSED
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 57339: Server startup script keeps waiting even if DB consistency has failed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57339/#review167984 --- Ship it! Ship It! - Laszlo Puskas On March 6, 2017, 2:54 p.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57339/ > --- > > (Updated March 6, 2017, 2:54 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Andrew Onischuk, > Dmytro Sen, Laszlo Puskas, Nate Cole, Oliver Szabo, Robert Levas, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-20319 > https://issues.apache.org/jira/browse/AMBARI-20319 > > > Repository: ambari > > > Description > --- > > When DB check failed during ambari start up, the script waited some 50 > seconds for the UI before timeout occured, even though the Java process had > exited. This is fixed. Also added an extra pintStackTrace() in cases when > unexpected error occured during startup. This is needed as testing showed > that logs are not always flushed on system exit and importan diagnostic > information was lost. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > 9540ca3 > ambari-server/src/main/python/ambari_server/utils.py 6408285 > ambari-server/src/main/python/ambari_server_main.py 0cd19cc > > > Diff: https://reviews.apache.org/r/57339/diff/1/ > > > Testing > --- > > - Tested manually. > - Run all python unit tests in ambari-server, all passed > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 57040: Rebalance HDFS operation returns after the command is issued
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/ --- (Updated March 3, 2017, 12:30 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- Updated to the patch against the trunk. Bugs: AMBARI-20175 https://issues.apache.org/jira/browse/AMBARI-20175 Repository: ambari Description --- The rebalancing operation may take a long time (hours, days) thus when issued from the ambari UI the background operation may time out. As it's not possible to dynamically predict how long the rebalancing will last , the approach taken by this solution is to only trigger the operation and not wait to the operation to finish. (NOTE: after this change the progress of the rebalance operation won't be tracked anymore in the background operation; also the user won't be notified about the success/failure of the operation) UPDATE: The operation is only triggered if there's no other rebalance operation running. (This is done by checking the existence of the /system/balancer.id on hdfs As per the input from hdfs devs, this check may be misleading if a balancer process is killed by an admin, as the file can remain there. - To overcome this added a message to explain this behavior) Diffs (updated) - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_rebalance.py 1dc545e ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py bd05a95 ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_rebalance.py 1dc545e ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 08578bc ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py ec19dc4 Diff: https://reviews.apache.org/r/57040/diff/4/ Changes: https://reviews.apache.org/r/57040/diff/3-4/ Testing --- Manually tested on local unsecure cluster. Manually tested on local secure cluster Unit tests successfully ran. Thanks, Laszlo Puskas
Re: Review Request 57040: Rebalance HDFS operation returns after the command is issued
> On Feb. 24, 2017, 4:54 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py > > Line 346 (original), 346 (patched) > > <https://reviews.apache.org/r/57040/diff/2/?file=1647188#file1647188line346> > > > > Please also make this change for HDFS 3.0.0.3.0 Added patch for the trunk to the jira. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/#review166728 --- On March 3, 2017, 8:52 a.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57040/ > --- > > (Updated March 3, 2017, 8:52 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, > and Sebastian Toader. > > > Bugs: AMBARI-20175 > https://issues.apache.org/jira/browse/AMBARI-20175 > > > Repository: ambari > > > Description > --- > > The rebalancing operation may take a long time (hours, days) thus when issued > from the ambari UI the background operation may time out. > As it's not possible to dynamically predict how long the rebalancing will > last , the approach taken by this solution is to only trigger the operation > and not wait to the operation to finish. > > (NOTE: after this change the progress of the rebalance operation won't be > tracked anymore in the background operation; also the user won't be notified > about the success/failure of the operation) > > UPDATE: > The operation is only triggered if there's no other rebalance operation > running. (This is done by checking the existence of the /system/balancer.id > on hdfs As per the input from hdfs devs, this check may be misleading if a > balancer process is killed by an admin, as the file can remain there. - To > overcome this added a message to explain this behavior) > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_rebalance.py > 1dc545e > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py > 123486e > ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py fae500f > > > Diff: https://reviews.apache.org/r/57040/diff/3/ > > > Testing > --- > > Manually tested on local unsecure cluster. > Manually tested on local secure cluster > > Unit tests successfully ran. > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 57040: Rebalance HDFS operation returns after the command is issued
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/ --- (Updated March 3, 2017, 8:52 a.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20175 https://issues.apache.org/jira/browse/AMBARI-20175 Repository: ambari Description --- The rebalancing operation may take a long time (hours, days) thus when issued from the ambari UI the background operation may time out. As it's not possible to dynamically predict how long the rebalancing will last , the approach taken by this solution is to only trigger the operation and not wait to the operation to finish. (NOTE: after this change the progress of the rebalance operation won't be tracked anymore in the background operation; also the user won't be notified about the success/failure of the operation) UPDATE: The operation is only triggered if there's no other rebalance operation running. (This is done by checking the existence of the /system/balancer.id on hdfs As per the input from hdfs devs, this check may be misleading if a balancer process is killed by an admin, as the file can remain there. - To overcome this added a message to explain this behavior) Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_rebalance.py 1dc545e ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 123486e ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py fae500f Diff: https://reviews.apache.org/r/57040/diff/3/ Testing (updated) --- Manually tested on local unsecure cluster. Manually tested on local secure cluster Unit tests successfully ran. Thanks, Laszlo Puskas
Re: Review Request 57040: Rebalance HDFS operation returns after the command is issued
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/ --- (Updated March 3, 2017, 8:16 a.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- Added check for another balancer running. Added message (to the backgroung operation popup) to explain how the command operates. Bugs: AMBARI-20175 https://issues.apache.org/jira/browse/AMBARI-20175 Repository: ambari Description (updated) --- The rebalancing operation may take a long time (hours, days) thus when issued from the ambari UI the background operation may time out. As it's not possible to dynamically predict how long the rebalancing will last , the approach taken by this solution is to only trigger the operation and not wait to the operation to finish. (NOTE: after this change the progress of the rebalance operation won't be tracked anymore in the background operation; also the user won't be notified about the success/failure of the operation) UPDATE: The operation is only triggered if there's no other rebalance operation running. (This is done by checking the existence of the /system/balancer.id on hdfs As per the input from hdfs devs, this check may be misleading if a balancer process is killed by an admin, as the file can remain there. - To overcome this added a message to explain this behavior) Diffs (updated) - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_rebalance.py 1dc545e ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 123486e ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py fae500f Diff: https://reviews.apache.org/r/57040/diff/3/ Changes: https://reviews.apache.org/r/57040/diff/2-3/ Testing (updated) --- Manually tested on local unsecure cluster. Manually tested on local secure cluster Unit tests running. Thanks, Laszlo Puskas
Re: Review Request 57147: Added support for processing custom command script timeout
> On March 1, 2017, 7:07 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Lines 64 (patched) > > <https://reviews.apache.org/r/57147/diff/1/?file=1651734#file1651734line64> > > > > I'm still opposed to this patch fundamentally because no command should > > be able to run for 2 hours. That will block other commands on the client > > that decided to run it, which is extremely problematic. > > > > Please make this call return right away and check the existence of the > > /system/balancer.id file OK, i'll drop this patch. The rebalance operation will return imediately after the command is issued; in case nother balancer is running (the above file exists on hdfs). there will be no feedback about the success/failure of the command. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57147/#review167547 ------- On Feb. 28, 2017, 6:01 p.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57147/ > --- > > (Updated Feb. 28, 2017, 6:01 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, > and Sebastian Toader. > > > Bugs: AMBARI-20175 > https://issues.apache.org/jira/browse/AMBARI-20175 > > > Repository: ambari > > > Description > --- > > HDFS rebalance operation can last for a long time (hours/days) thus when > triggered from the UI the command may be timed out by the Ambari server. > This behavior may confuse users, making them to trigger the rebalancer again > which will fail with "another balancer is running" error. > > The patch provides support for setting a reasonably long timeout for the > rebalance custo action so that Ambari server doesnt time out the command. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > 867ebff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > fd7f2f6 > > > Diff: https://reviews.apache.org/r/57147/diff/1/ > > > Testing > --- > > Unit tests OK. > Manually tested on local dev-env. > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 57147: Added support for processing custom command script timeout
> On Feb. 28, 2017, 6:32 p.m., Alejandro Fernandez wrote: > > Thanks Alejanro, i wasn't ware of that; I'll do the modification to the other approach then. > On Feb. 28, 2017, 6:32 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Lines 64 (patched) > > <https://reviews.apache.org/r/57147/diff/1/?file=1651734#file1651734line64> > > > > What happens if a rebalance command is issued and one is already > > running? > > > > I don't want a process running on Ambari for up to 2 hours. And for > > very large clusters (1000+ nodes), it may still timeout. > > Alejandro Fernandez wrote: > HDFS stores a file in /system/balancer.id while the balancer is on-going. > As soon as it finishes, it is deleted. > You can use that to make the call return right away and run in the > background, and also prevent subsequent calls from launching another > rebalancer. This is intended to be a workaround as the current implementation caused problems when the rebalancing process took a long time. While i also think that having long running custom command is not ideal, this approach seemed to be the right compromise on having some feedback from the process. If two users trigger/start the rebalancing separately, the second will fail with "another balancer is running", as it happens today. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57147/#review167117 --- On Feb. 28, 2017, 6:01 p.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57147/ > --- > > (Updated Feb. 28, 2017, 6:01 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, > and Sebastian Toader. > > > Bugs: AMBARI-20175 > https://issues.apache.org/jira/browse/AMBARI-20175 > > > Repository: ambari > > > Description > --- > > HDFS rebalance operation can last for a long time (hours/days) thus when > triggered from the UI the command may be timed out by the Ambari server. > This behavior may confuse users, making them to trigger the rebalancer again > which will fail with "another balancer is running" error. > > The patch provides support for setting a reasonably long timeout for the > rebalance custo action so that Ambari server doesnt time out the command. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > 867ebff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > fd7f2f6 > > > Diff: https://reviews.apache.org/r/57147/diff/1/ > > > Testing > --- > > Unit tests OK. > Manually tested on local dev-env. > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 57147: Added support for processing custom command script timeout
> On Feb. 28, 2017, 6:06 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml, > > line 64 > > <https://reviews.apache.org/r/57147/diff/1/?file=1651734#file1651734line64> > > > > I don't think Ambari should run a command that takes 2 hours! > > Otherwise, the user will not be able to run other commands on that host. > > > > Instead, we need HDFS to return right away and indicate if a rebalance > > is already happening. > > Alejandro Fernandez wrote: > Please ask for input from the HDFS team on how to best do this. We asked them already, they confirmed, that apart of the command output it's not possible to track/retrieve the status of the balancer. So once we loose the connection with the command output we can't get information about the balancer. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57147/#review167110 ----------- On Feb. 28, 2017, 6:01 p.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57147/ > --- > > (Updated Feb. 28, 2017, 6:01 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Robert Levas, Sandor Magyari, > and Sebastian Toader. > > > Bugs: AMBARI-20175 > https://issues.apache.org/jira/browse/AMBARI-20175 > > > Repository: ambari > > > Description > --- > > HDFS rebalance operation can last for a long time (hours/days) thus when > triggered from the UI the command may be timed out by the Ambari server. > This behavior may confuse users, making them to trigger the rebalancer again > which will fail with "another balancer is running" error. > > The patch provides support for setting a reasonably long timeout for the > rebalance custo action so that Ambari server doesnt time out the command. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > 867ebff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > fd7f2f6 > > Diff: https://reviews.apache.org/r/57147/diff/ > > > Testing > --- > > Unit tests OK. > Manually tested on local dev-env. > > > Thanks, > > Laszlo Puskas > >
Re: Review Request 57040: Rebalance HDFS operation returns after the command is issued
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/ --- (Updated Feb. 24, 2017, 4:34 p.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20175 https://issues.apache.org/jira/browse/AMBARI-20175 Repository: ambari Description --- The rebalancing operation may take a long time (hours, days) thus when issued from the ambari UI the background operation may time out. As it's not possible to dynamically predict how long the rebalancing will last , the approach taken by this solution is to only trigger the operation and not wait to the operation to finish. (NOTE: after this change the progress of the rebalance operation won't be tracked anymore in the background operation; also the user won't be notified about the success/failure of the operation) Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 123486e ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py fae500f Diff: https://reviews.apache.org/r/57040/diff/ Testing (updated) --- Manually. Unit tests success. Thanks, Laszlo Puskas
Re: Review Request 57040: Rebalance HDFS operation returns after the command is issued
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/ --- (Updated Feb. 24, 2017, 4:18 p.m.) Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Changes --- Fixed unit tests. Bugs: AMBARI-20175 https://issues.apache.org/jira/browse/AMBARI-20175 Repository: ambari Description --- The rebalancing operation may take a long time (hours, days) thus when issued from the ambari UI the background operation may time out. As it's not possible to dynamically predict how long the rebalancing will last , the approach taken by this solution is to only trigger the operation and not wait to the operation to finish. (NOTE: after this change the progress of the rebalance operation won't be tracked anymore in the background operation; also the user won't be notified about the success/failure of the operation) Diffs (updated) - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 123486e ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py fae500f Diff: https://reviews.apache.org/r/57040/diff/ Testing (updated) --- Manually. Unit tests running... Thanks, Laszlo Puskas
Review Request 57040: Rebalance HDFS operation returns after the command is issued
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57040/ --- Review request for Ambari, Robert Levas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-20175 https://issues.apache.org/jira/browse/AMBARI-20175 Repository: ambari Description --- The rebalancing operation may take a long time (hours, days) thus when issued from the ambari UI the background operation may time out. As it's not possible to dynamically predict how long the rebalancing will last , the approach taken by this solution is to only trigger the operation and not wait to the operation to finish. (NOTE: after this change the progress of the rebalance operation won't be tracked anymore in the background operation; also the user won't be notified about the success/failure of the operation) Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 123486e Diff: https://reviews.apache.org/r/57040/diff/ Testing --- Manually. Unit tests running. Thanks, Laszlo Puskas
Re: Review Request 56783: Removing secure ACLs from Kafka znodes during dekerberization
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56783/#review165942 --- Ship it! Ship It! - Laszlo Puskas On Feb. 17, 2017, 3:16 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56783/ > --- > > (Updated Feb. 17, 2017, 3:16 p.m.) > > > Review request for Ambari, Laszlo Puskas, Robert Levas, and Sebastian Toader. > > > Bugs: AMBARI-20063 > https://issues.apache.org/jira/browse/AMBARI-20063 > > > Repository: ambari > > > Description > --- > > After enabling kerberos kafka sets secure ACLs on certain znodes. These > secure ACLs should be removed by ambari during dekerberization. > > Note that the znode names are hardcoded in the params.py because AFAIK they > are also hardcoded in kafka. I'm still waiting for a kafka developer to > confirm this. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka_broker.py > 0901730 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py > 1d3a195 > > Diff: https://reviews.apache.org/r/56783/diff/ > > > Testing > --- > > Manually: > - created cluster with kafka > - enabled kerberos > - checked that acls on kafka znodes are secure > - disabled kerberos > - checked that acls on kafka znodes are 'world:anyone:crdwa' > > Ran 271 tests in 7.521s > OK > -- > Total run:1173 > Total errors:0 > Total failures:0 > -- > Ran 451 tests in 99.286s > OK > Results : > Tests run: 4925, Failures: 0, Errors: 0, Skipped: 39 > > > Thanks, > > Attila Magyar > >
Re: Review Request 56713: Ambari server start returns prematurely before extracting views.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56713/#review165719 --- Ship it! Ship It! - Laszlo Puskas On Feb. 15, 2017, 3:24 p.m., Dmytro Sen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56713/ > --- > > (Updated Feb. 15, 2017, 3:24 p.m.) > > > Review request for Ambari, Laszlo Puskas, Sebastian Toader, Sid Wagle, and > Vitalyi Brodetskyi. > > > Bugs: AMBARI-20026 > https://issues.apache.org/jira/browse/AMBARI-20026 > > > Repository: ambari > > > Description > --- > > When ambari-server start is issued, It returns prematurely before the view > extraction is complete. > because of this we have introduced sleep of 3mins before ambari-server stop > for the view extraction to complete. > Ambari server should report start completed only after view extraction is > completed > > > Diffs > - > > ambari-server/src/main/python/ambari_server/serverConfiguration.py d3bfa70 > ambari-server/src/main/python/ambari_server/utils.py f65ceed > ambari-server/src/main/python/ambari_server_main.py df1b0a1 > ambari-server/src/test/python/TestAmbariServer.py 8692030 > > Diff: https://reviews.apache.org/r/56713/diff/ > > > Testing > --- > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [4.125s] > [INFO] Apache Ambari Project POM . SUCCESS [0.006s] > [INFO] utility ... SUCCESS [2.523s] > [INFO] Ambari Agent .. SUCCESS [34.664s] > [INFO] > > [INFO] BUILD SUCCESS > > > Thanks, > > Dmytro Sen > >
Re: Review Request 56599: ambari-server upgrade is not idempotent
> On Feb. 13, 2017, 3:08 p.m., Laszlo Puskas wrote: > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java, > > line 1672 > > <https://reviews.apache.org/r/56599/diff/1/?file=1631931#file1631931line1672> > > > > Instead mocking tehe injector itself, add a module where instances are > > bound to mocks. > > Attila Magyar wrote: > Tried it but didn't work, there are other dependencies that should be > initialized. > > Laszlo Puskas wrote: > Dropped as binding classes to instances cause the framework to interpret annotations and tries to inject all dependencies. - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56599/#review165330 --- On Feb. 13, 2017, 1:48 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56599/ > --- > > (Updated Feb. 13, 2017, 1:48 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Robert > Levas, and Sebastian Toader. > > > Bugs: AMBARI-19984 > https://issues.apache.org/jira/browse/AMBARI-19984 > > > Repository: ambari > > > Description > --- > > Running ambari-upgrade multiple times cause an exception due to violation of > unique database constraint (the same RoleAuthorization can be added multiple > times to admin permissions). > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PermissionEntity.java > fb01cca > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 4b33bcd > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog242Test.java > 0cd4f12f > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java > 1c742ef > > Diff: https://reviews.apache.org/r/56599/diff/ > > > Testing > --- > > Added new unittest that checks if adding RoleAuthorization is idempotent. > > Tested manually: > - installed ambari 2.4 > - upgraded to ambari 2.5 > - set version back to 2.4 by running: update metainfo set metainfo_value = > '2.4.0' where metainfo_key = 'version'; > - upgraded to ambari 2.5 again > - started ambari-server checked the UI > > > Existing tests ran clealy > > -- > Ran 465 tests in 15.977s > > Tests run: 4915, Failures: 0, Errors: 0, Skipped: 39 > > > Thanks, > > Attila Magyar > >
Re: Review Request 56599: ambari-server upgrade is not idempotent
> On Feb. 13, 2017, 3:08 p.m., Laszlo Puskas wrote: > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java, > > line 1672 > > <https://reviews.apache.org/r/56599/diff/1/?file=1631931#file1631931line1672> > > > > Instead mocking tehe injector itself, add a module where instances are > > bound to mocks. > > Attila Magyar wrote: > Tried it but didn't work, there are other dependencies that should be > initialized. Dropped as binding classes to instances cause the framework to interpret - Laszlo --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56599/#review165330 --- On Feb. 13, 2017, 1:48 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56599/ > --- > > (Updated Feb. 13, 2017, 1:48 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Robert > Levas, and Sebastian Toader. > > > Bugs: AMBARI-19984 > https://issues.apache.org/jira/browse/AMBARI-19984 > > > Repository: ambari > > > Description > --- > > Running ambari-upgrade multiple times cause an exception due to violation of > unique database constraint (the same RoleAuthorization can be added multiple > times to admin permissions). > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PermissionEntity.java > fb01cca > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 4b33bcd > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog242Test.java > 0cd4f12f > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java > 1c742ef > > Diff: https://reviews.apache.org/r/56599/diff/ > > > Testing > --- > > Added new unittest that checks if adding RoleAuthorization is idempotent. > > Tested manually: > - installed ambari 2.4 > - upgraded to ambari 2.5 > - set version back to 2.4 by running: update metainfo set metainfo_value = > '2.4.0' where metainfo_key = 'version'; > - upgraded to ambari 2.5 again > - started ambari-server checked the UI > > > Existing tests ran clealy > > -- > Ran 465 tests in 15.977s > > Tests run: 4915, Failures: 0, Errors: 0, Skipped: 39 > > > Thanks, > > Attila Magyar > >
Re: Review Request 56599: ambari-server upgrade is not idempotent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56599/#review165330 --- ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java (line 1666) <https://reviews.apache.org/r/56599/#comment237174> Instead mocking tehe injector itself, add a module where instances are bound to mocks. - Laszlo Puskas On Feb. 13, 2017, 1:48 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56599/ > --- > > (Updated Feb. 13, 2017, 1:48 p.m.) > > > Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, Robert > Levas, and Sebastian Toader. > > > Bugs: AMBARI-19984 > https://issues.apache.org/jira/browse/AMBARI-19984 > > > Repository: ambari > > > Description > --- > > Running ambari-upgrade multiple times cause an exception due to violation of > unique database constraint (the same RoleAuthorization can be added multiple > times to admin permissions). > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PermissionEntity.java > fb01cca > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 4b33bcd > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog242Test.java > 0cd4f12f > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java > 1c742ef > > Diff: https://reviews.apache.org/r/56599/diff/ > > > Testing > --- > > Added new unittest that checks if adding RoleAuthorization is idempotent. > > Tested manually: > - installed ambari 2.4 > - upgraded to ambari 2.5 > - set version back to 2.4 by running: update metainfo set metainfo_value = > '2.4.0' where metainfo_key = 'version'; > - upgraded to ambari 2.5 again > - started ambari-server checked the UI > > > Existing tests ran clealy > > -- > Ran 465 tests in 15.977s > > Tests run: 4915, Failures: 0, Errors: 0, Skipped: 39 > > > Thanks, > > Attila Magyar > >
Review Request 56600: Default group permissions for the user home directories created by the post user creation script should be 'hdfs' instead of 'hadoop'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56600/ --- Review request for Ambari, Attila Doroszlai, Attila Magyar, and Sebastian Toader. Bugs: AMBARI-19986 https://issues.apache.org/jira/browse/AMBARI-19986 Repository: ambari Description --- Default group permissions for the user home directories created by the post user creation script should be 'hdfs' instead of 'hadoop' Diffs - ambari-server/src/main/resources/scripts/post-user-creation-hook.sh 91511a0 Diff: https://reviews.apache.org/r/56600/diff/ Testing --- Manually, underway. Thanks, Laszlo Puskas
Re: Review Request 56175: Setup correct authentication and authorization mechanism between Yarn Registry and Zookeeper
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56175/#review163842 --- Ship it! Ship It! - Laszlo Puskas On Feb. 1, 2017, 4:43 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56175/ > --- > > (Updated Feb. 1, 2017, 4:43 p.m.) > > > Review request for Ambari, Laszlo Puskas, Robert Levas, and Sebastian Toader. > > > Bugs: AMBARI-19331 > https://issues.apache.org/jira/browse/AMBARI-19331 > > > Repository: ambari > > > Description > --- > > YARN registry and yarn leader election znodes must be protected by sasl ACLs > when kerberos enabled. The sasl ACLs should be removed after disabling > kerberos. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py > aed8abc > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py > a659dd1 > > ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/kerberos.json > 29cc00a > > ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py > 4d47925 > > ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/resourcemanager.py > 4d8d95e > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/kerberos.json > eaffec6 > > Diff: https://reviews.apache.org/r/56175/diff/ > > > Testing > --- > > Manual testing: > - created a cluster with yarn and hive > - enabled hive interactive mode > - enabled kerberos > - checked that hive interactive started successfully > - checked acls on /registry > - disabled kerberos > - checked acls on /registry > > Unittest run successfully (1 unrelated failure) > > Failed tests: > > AsyncCallableServiceTest.testCallableServiceShouldCancelTaskWhenTimeoutExceeded:94 > > Unexpected method call ScheduledExecutorService.schedule(EasyMock for > interface java.util.concurrent.Callable, 500, MILLISECONDS): > > Tests run: 4884, Failures: 1, Errors: 0, Skipped: 39 > Ran 270 tests in 6.748s > > > ambari-agent: >Ran 451 tests in 95.436s > > > Thanks, > > Attila Magyar > >
Re: Review Request 56064: Format ZKFC commands failing while enabling NameNode HA
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56064/#review163483 --- Ship it! Ship It! - Laszlo Puskas On Jan. 29, 2017, 8:43 p.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56064/ > --- > > (Updated Jan. 29, 2017, 8:43 p.m.) > > > Review request for Ambari, Attila Magyar, Laszlo Puskas, and Robert Levas. > > > Bugs: AMBARI-19736 > https://issues.apache.org/jira/browse/AMBARI-19736 > > > Repository: ambari > > > Description > --- > > Format ZKFC commands failing while enabling NameNode HA at Initializa > Metadata step. > > ``` > Caused by: org.apache.zookeeper.KeeperException$NoAuthException: > KeeperErrorCode = NoAuth for /hadoop-ha/nameservice > at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) > at > org.apache.hadoop.ha.ActiveStandbyElector$3.run(ActiveStandbyElector.java:1000) > at > org.apache.hadoop.ha.ActiveStandbyElector$3.run(ActiveStandbyElector.java:997) > at > org.apache.hadoop.ha.ActiveStandbyElector.zkDoWithRetries(ActiveStandbyElector.java:1041) > at > org.apache.hadoop.ha.ActiveStandbyElector.createWithRetries(ActiveStandbyElector.java:997) > at > org.apache.hadoop.ha.ActiveStandbyElector.ensureParentZNode(ActiveStandbyElector.java:344) > ... 11 more > ``` > > The reason for the failure is that the `hdfs_jaas.conf` file is generated > during ZKFC component configuration. When NN HA is enabled via UI the ZKFC is > not added yet to the cluster thus the `hdfs_jaas.conf` file is not generated > yet, this leading the format ZKFC commands to fail as this require the jaas > file. > > The solution is to move the creation of `hdfs_jaas.conf` file into the NN > configuration. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py > a2edf38 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py > 03aba7b > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py > bfc9429 > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py > 7fae57f > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py > 9eebe63 > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/zkfc_slave.py > f2ea6ad > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py > 8e0e783 > > ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/params.py > 8e5d210 > > Diff: https://reviews.apache.org/r/56064/diff/ > > > Testing > --- > > Manual testing using both wizzard and blueprints. > > Unit tests: > ``` > mvn test -DskipSurefireTests -am -pl ambari-server > > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [11.634s] > [INFO] Apache Ambari Project POM . SUCCESS [0.045s] > [INFO] Ambari Views .. SUCCESS [2.086s] > [INFO] utility ... SUCCESS [1.175s] > [INFO] ambari-metrics SUCCESS [0.608s] > [INFO] Ambari Metrics Common . SUCCESS [0.352s] > [INFO] Ambari Server . SUCCESS [2:25.628s] > [INFO] > > [INFO] BUILD SUCCESS > ``` > > > Thanks, > > Sebastian Toader > >
Re: Review Request 56054: Inconsistent auth-to-local rules processing during Kerberos authentication
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56054/#review163430 --- Ship it! Ship It! - Laszlo Puskas On Jan. 29, 2017, 12:50 a.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56054/ > --- > > (Updated Jan. 29, 2017, 12:50 a.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Laszlo Puskas, and Sebastian Toader. > > > Bugs: AMBARI-19767 > https://issues.apache.org/jira/browse/AMBARI-19767 > > > Repository: ambari > > > Description > --- > > Facing issue with local to auth rules. > ambari-qa-...@example.com is getting converted to ambari-qa-cl1 as well as > ambari-qa with same ambari configuration ie > authentication.kerberos.auth_to_local.rules=DEFAULT. > > 1st translation : > ``` > 28 Jan 2017 11:44:45,529 INFO [ambari-client-thread-3298] > AmbariAuthToLocalUserDetailsService:102 - Translated > ambari-qa-...@example.com to ambari-qa-cl1 using auth-to-local rules during > Kerberos authentication. > ``` > > 2nd translation : > ``` > 28 Jan 2017 11:47:36,425 INFO [ambari-client-thread-3172] > AmbariAuthToLocalUserDetailsService:102 - Translated > ambari-qa-...@example.com to ambari-qa using auth-to-local rules during > Kerberos authentication. > 28 Jan 2017 11:47:36,428 WARN [ambari-client-thread-3172] > AmbariAuthToLocalUserDetailsService:136 - Failed find user account for user > with username of ambari-qa during Kerberos authentication. > 28 > ``` > > Since authentication.kerberos.auth_to_local.rules=DEFAULT , > 'ambari-qa-...@example.com' should have been translated to 'ambari-qa-cl1'. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariAuthToLocalUserDetailsService.java > c85503c > > Diff: https://reviews.apache.org/r/56054/diff/ > > > Testing > --- > > # Local test results: > > ``` > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 35:42.972s > [INFO] Finished at: Sat Jan 28 17:37:32 EST 2017 > [INFO] Final Memory: 66M/1180M > [INFO] > > ``` > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
