Re: Review Request 64971: AMBARI-22252. Revert Blueprint V2 changes
ures: 2, Errors: 8, Skipped: 0 - in org.apache.ambari.server.topology.TopologyManagerTest +Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.TopologyManagerTest ``` Thanks, Attila Doroszlai
Re: Review Request 64985: AMBARI-22244. Use service type instead of service name - addendum
> On Jan. 5, 2018, 10:57 p.m., Madhuvanthi Radhakrishnan wrote: > > Are we using serviceType because for 3.0, we will have configurations per > > serviceType and not per serviceInstance? No, these methods provide properties defined in the stack, ie. not service instances but service types. So they only know about eg. ZOOKEEPER, not ZK1 and ZK2. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64985/#review194906 --- On Jan. 5, 2018, 8:10 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64985/ > --- > > (Updated Jan. 5, 2018, 8:10 p.m.) > > > Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Madhuvanthi > Radhakrishnan, and Robert Nettleton. > > > Bugs: AMBARI-22244 > https://issues.apache.org/jira/browse/AMBARI-22244 > > > Repository: ambari > > > Description > --- > > cherry-picked from `branch-feature-AMBARI-14714-blueprintv2` branch: > > Replaced `serviceName` with `serviceType` in more places > (`getDefaultProperties`, `getServiceProperties` and `getReadOnlyProperties` > calls). > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java > 2582c71eb95255f0f8081e90473183243f12cdd0 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java > 38e1233308a7f423789a1961e3dd8664919f29ae > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java > 6bb248e19bef39ba2edb3671d38ef815d8efb593 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > 4008c21f8ef092e2a1ebd0d7c18ad95bc8206e37 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java > 36918ccfef596e14a0d3e8e663fb097d065e27a4 > > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java > a6c650705ad6b90834906bd3ce08149e2528f28e > > > Diff: https://reviews.apache.org/r/64985/diff/1/ > > > Testing > --- > > Used in blueprint cluster creation (on > `branch-feature-AMBARI-14714-blueprintv2` branch). > > > Thanks, > > Attila Doroszlai > >
Review Request 64985: AMBARI-22244. Use service type instead of service name - addendum
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64985/ --- Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Madhuvanthi Radhakrishnan, and Robert Nettleton. Bugs: AMBARI-22244 https://issues.apache.org/jira/browse/AMBARI-22244 Repository: ambari Description --- cherry-picked from `branch-feature-AMBARI-14714-blueprintv2` branch: Replaced `serviceName` with `serviceType` in more places (`getDefaultProperties`, `getServiceProperties` and `getReadOnlyProperties` calls). Diffs - ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java 2582c71eb95255f0f8081e90473183243f12cdd0 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 38e1233308a7f423789a1961e3dd8664919f29ae ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 6bb248e19bef39ba2edb3671d38ef815d8efb593 ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 4008c21f8ef092e2a1ebd0d7c18ad95bc8206e37 ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java 36918ccfef596e14a0d3e8e663fb097d065e27a4 ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a6c650705ad6b90834906bd3ce08149e2528f28e Diff: https://reviews.apache.org/r/64985/diff/1/ Testing --- Used in blueprint cluster creation (on `branch-feature-AMBARI-14714-blueprintv2` branch). Thanks, Attila Doroszlai
Re: Review Request 64976: Ambari Infra Manager: hdfs upload support
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64976/#review194865 --- Ship it! ambari-infra/ambari-infra-manager/docker/infra-manager-docker-compose.sh Lines 91-94 (original), 99-104 (patched) <https://reviews.apache.org/r/64976/#comment273945> Is there any reason for not using `docker-compose rm -f` instead? - Attila Doroszlai On Jan. 5, 2018, 3:43 p.m., Krisztian Kasa wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64976/ > --- > > (Updated Jan. 5, 2018, 3:43 p.m.) > > > Review request for Ambari, Attila Doroszlai, Oliver Szabo, and Sid Wagle. > > > Bugs: AMBARI-22731 > https://issues.apache.org/jira/browse/AMBARI-22731 > > > Repository: ambari > > > Description > --- > > * while arhiving documents from solr result files can be saved to hdfs > * s3 and hdfs releated parameters can be overwritten using infra manger api > * destination directory name contains the interval end value when archiving > to local filesystem > * job can be restarted using infra manger api > * bugfix: infra manger startup failed when no jobs were defined in the > properties > * IT: test for archiving to hdfs > * IT: test for archiving to local filesystem > * vagrant-infra-manager.properties file for integration testing with an > ambari installed cluster > > > Diffs > - > > ambari-infra/ambari-infra-manager-it/pom.xml 97e8ea0f44 > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java > 0e391a3e6e > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/steps/AbstractInfraSteps.java > f48d4c2b49 > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/steps/ExportJobsSteps.java > 22826a01fc > > ambari-infra/ambari-infra-manager-it/src/test/resources/stories/infra_api_tests.story > 1044258816 > ambari-infra/ambari-infra-manager/docker/docker-compose.yml 1172631917 > ambari-infra/ambari-infra-manager/docker/infra-manager-docker-compose.sh > 6ddb7c292a > ambari-infra/ambari-infra-manager/pom.xml de131b03f6 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/JobProperties.java > 0841dd7fd9 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentArchivingConfiguration.java > 7588b9992a > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportProperties.java > 1484eedfe3 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/ExportDestination.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/FileNameSuffixFormatter.java > af48ab9aaa > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/HdfsUploader.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/deleting/DocumentDeletingConfiguration.java > 4fce4b677f > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/model/JobExecutionRestartRequest.java > 88687e7260 > > ambari-infra/ambari-infra-manager/src/main/resources/infra-manager.properties > 9103d09e7e > > ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/JobPropertiesTest.java > 6a7622954f > > ambari-infra/ambari-infra-manager/src/test/java/org/apache/ambari/infra/job/archive/FileNameSuffixFormatterTest.java > 34e679f298 > > ambari-infra/ambari-infra-manager/src/test/resoruces/vagrant-infra-manager.properties > PRE-CREATION > ambari-infra/pom.xml 9e7a71bfa7 > > > Diff: https://reviews.apache.org/r/64976/diff/1/ > > > Testing > --- > > * unit tests > * integration test using docker > * integration test using vagrant, ambari and ide > > > Thanks, > > Krisztian Kasa > >
Review Request 64971: AMBARI-22252. Revert Blueprint V2 changes
3ec80 Diff: https://reviews.apache.org/r/64971/diff/1/ Testing --- Deployed HDP-2.6 cluster via "old" blueprint. Tweaked `AmbariContextTest` to verify service group creation. Checks pass: ``` [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 6282 licence. ... [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server --- [INFO] Starting audit... Audit done. ``` Unit tests improved from: ``` Tests run: 4965, Failures: 192, Errors: 283, Skipped: 37 ``` to: ``` Tests run: 4964, Failures: 37, Errors: 179, Skipped: 37 ``` Specifically: ``` -Tests run: 168, Failures: 153, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest +Tests run: 168, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest -Tests run: 27, Failures: 0, Errors: 8, Skipped: 0 - in org.apache.ambari.server.controller.internal.BlueprintResourceProviderTest +Tests run: 27, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.BlueprintResourceProviderTest -Tests run: 21, Failures: 0, Errors: 1, Skipped: 0 - in org.apache.ambari.server.controller.internal.ClusterResourceProviderTest +Tests run: 21, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.ClusterResourceProviderTest -Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 - in org.apache.ambari.server.controller.internal.ExportBlueprintRequestTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.ExportBlueprintRequestTest -Tests run: 33, Failures: 17, Errors: 16, Skipped: 0 - in org.apache.ambari.server.controller.internal.ProvisionClusterRequestTest +Tests run: 17, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.ProvisionClusterRequestTest -Tests run: 16, Failures: 6, Errors: 10, Skipped: 0 - in org.apache.ambari.server.controller.internal.ScaleClusterRequestTest +Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.ScaleClusterRequestTest -Tests run: 7, Failures: 0, Errors: 7, Skipped: 0 - in org.apache.ambari.server.controller.internal.UnitUpdaterTest +Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.controller.internal.UnitUpdaterTest -Tests run: 30, Failures: 0, Errors: 30, Skipped: 0 - in org.apache.ambari.server.topology.AmbariContextTest +Tests run: 15, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.AmbariContextTest -Tests run: 11, Failures: 0, Errors: 2, Skipped: 0 - in org.apache.ambari.server.topology.BlueprintImplTest +Tests run: 11, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.BlueprintImplTest -Tests run: 9, Failures: 0, Errors: 9, Skipped: 0 - in org.apache.ambari.server.topology.ClusterConfigurationRequestTest +Tests run: 9, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.ClusterConfigurationRequestTest -Tests run: 2, Failures: 1, Errors: 1, Skipped: 0 - in org.apache.ambari.server.topology.ClusterDeployWithStartOnlyTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.ClusterDeployWithStartOnlyTest -Tests run: 2, Failures: 1, Errors: 1, Skipped: 0 - in org.apache.ambari.server.topology.ClusterInstallWithoutStartOnComponentLevelTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.ClusterInstallWithoutStartOnComponentLevelTest -Tests run: 2, Failures: 1, Errors: 1, Skipped: 0 - in org.apache.ambari.server.topology.ClusterInstallWithoutStartTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.ClusterInstallWithoutStartTest -Tests run: 10, Failures: 4, Errors: 5, Skipped: 0 - in org.apache.ambari.server.topology.ClusterTopologyImplTest +Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.ClusterTopologyImplTest -Tests run: 8, Failures: 1, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.HostGroupInfoTest +Tests run: 8, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.HostGroupInfoTest -Tests run: 7, Failures: 0, Errors: 7, Skipped: 0 - in org.apache.ambari.server.topology.LogicalRequestTest +Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.LogicalRequestTest -Tests run: 12, Failures: 0, Errors: 12, Skipped: 0 - in org.apache.ambari.server.topology.RequiredPasswordValidatorTest +Tests run: 12, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.RequiredPasswordValidatorTest -Tests run: 12, Failures: 2, Errors: 8, Skipped: 0 - in org.apache.ambari.server.topology.TopologyManagerTest +Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.topology.TopologyManagerTest ``` Thanks, Attila Doroszlai
Review Request 64914: AMBARI-22719. Regenerate Keytabs resets security.inter.broker.protocol to PLAINTEXTSASL
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64914/ --- Review request for Ambari, Attila Magyar, Balázs Bence Sári, Robert Levas, and Yolanda Davis. Bugs: AMBARI-22719 https://issues.apache.org/jira/browse/AMBARI-22719 Repository: ambari Description --- Let stack advisor retain `security.inter.broker.protocol` value if it is a secure one. Diffs - ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 125372011984791db4b1c54ef5874740882a2331 ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 4f8eaf2ceb985996b7e3d888e615c3bdc91f7c90 Diff: https://reviews.apache.org/r/64914/diff/1/ Testing --- Tested according to steps in [AMBARI-22719](https://issues.apache.org/jira/browse/AMBARI-22719). Amended existing unit test case. Unit tests are OK: ``` Total run:1200 Total errors:0 Total failures:0 OK ``` Thanks, Attila Doroszlai
Re: Review Request 64772: ServiceAdvisor for OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64772/#review194345 --- Fix it, then Ship it! Nice. contrib/management-packs/isilon-onefs-mpack/pom.xml Lines 33 (patched) <https://reviews.apache.org/r/64772/#comment273107> Looks like this doesn't work (due to different depth of project directory) and custom python path logic is required in `unitTests.py`. Can you please remove (also `pathsep`, and also their usage below) to avoid confusion? contrib/management-packs/isilon-onefs-mpack/pom.xml Lines 106 (patched) <https://reviews.apache.org/r/64772/#comment273108> This argument is actually ignored by `unitTests.py`. - Attila Doroszlai On Dec. 21, 2017, 3:46 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64772/ > --- > > (Updated Dec. 21, 2017, 3:46 p.m.) > > > Review request for Ambari, Attila Doroszlai, Jonathan Hurley, and Nate Cole. > > > Bugs: AMBARI-22674 > https://issues.apache.org/jira/browse/AMBARI-22674 > > > Repository: ambari > > > Description > --- > > A service advisor was added to the mpack to keep the hostnames of > core-site/fs.defaultFS, hdfs-site/dfs.namenode.http-address and > hdfs-site/dfs.namenode.https-address in sync. > > The user needs to enter the onefs/onefs_host and the service advisor will > update the other properties accordingly. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py 70f100f > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 5c0305e > ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py 803bdc6 > ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 02cc91c > ambari-agent/src/main/python/ambari_agent/alerts/web_alert.py 0e400f7 > ambari-common/src/main/python/ambari_commons/inet_utils.py 5fd3e88 > contrib/management-packs/isilon-onefs-mpack/pom.xml 5d8f215 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/configuration/core-site.xml > 7d3acd7 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/configuration/hdfs-site.xml > cb6544f > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/configuration/onefs.xml > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/metainfo.xml > 5ba562e > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/params_linux.py > 953efdc > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/service_advisor.py > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/themes/theme.json > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/test/python/TestServiceAdvisor.py > PRE-CREATION > contrib/management-packs/isilon-onefs-mpack/src/test/python/unitTests.py > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64772/diff/1/ > > > Testing > --- > > - cluster creation, changed configs manually > > existing tests: passed > > > Thanks, > > Attila Magyar > >
Re: Review Request 64747: Don't call pprint.pformat unnecessarily in Ambari agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64747/#review194232 --- Ship it! Ship It! - Attila Doroszlai On Dec. 20, 2017, 1:18 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64747/ > --- > > (Updated Dec. 20, 2017, 1:18 p.m.) > > > Review request for Ambari, Attila Doroszlai, Andrew Onischuk, and Eugene > Chekanskiy. > > > Bugs: AMBARI-22677 > https://issues.apache.org/jira/browse/AMBARI-22677 > > > Repository: ambari > > > Description > --- > > The pprint.pformat is relatively slow and ambari agent calls it frequently > just to generate debug log. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/ActionQueue.py f1b3a42 > ambari-agent/src/main/python/ambari_agent/CommandStatusDict.py 7a97f3f > ambari-agent/src/main/python/ambari_agent/Controller.py bc923c3 > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py > 645f824 > ambari-agent/src/main/python/ambari_agent/DataCleaner.py 0b66fa7 > ambari-agent/src/main/python/ambari_agent/Heartbeat.py d7c0325 > ambari-agent/src/main/python/ambari_agent/LiveStatus.py b6d54ed > ambari-agent/src/main/python/ambari_agent/PythonExecutor.py ea6f895 > ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py > b27d7d1 > ambari-agent/src/main/python/ambari_agent/RecoveryManager.py be335f2 > ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py f42e134 > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py 70f100f > ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py 803bdc6 > ambari-agent/src/main/python/ambari_agent/security.py 45de7bb > > > Diff: https://reviews.apache.org/r/64747/diff/1/ > > > Testing > --- > > ran unittests, created a cluster successfully with ambari > > > Thanks, > > Attila Magyar > >
Re: Review Request 64670: AMBARI-22660. Fix unit tests failing due to ServiceGroup issues
> On Dec. 18, 2017, noon, Balázs Bence Sári wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > > Line 625 (original), 629 (patched) > > <https://reviews.apache.org/r/64670/diff/1/?file=1919170#file1919170line629> > > > > Wonder if instead of swallowing an uncaught runtime exception we should > > rather modify the called method (getServiceByConfigType in ClusterImpl) to > > not throw this exception. Actually the runtime exception is now fixed by [an addendum commit](https://git1-us-west.apache.org/repos/asf?p=ambari.git;a=commit;h=bfeaa1071e). I'll remove the catch. > On Dec. 18, 2017, noon, Balázs Bence Sári wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java > > Line 64 (original), 64 (patched) > > <https://reviews.apache.org/r/64670/diff/1/?file=1919175#file1919175line64> > > > > This column will shortly be removed in another patch so I'd prefer not > > changing it. Sure. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64670/#review194055 --- On Dec. 17, 2017, 2:35 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64670/ > --- > > (Updated Dec. 17, 2017, 2:35 p.m.) > > > Review request for Ambari, Balázs Bence Sári, Dmytro Sen, Jayush Luniya, > Madhuvanthi Radhakrishnan, Robert Nettleton, Swapan Shridhar, and Vitalyi > Brodetskyi. > > > Bugs: AMBARI-22660 > https://issues.apache.org/jira/browse/AMBARI-22660 > > > Repository: ambari > > > Description > --- > > Fixed mostly `ServiceGroupNotFound` due to `serviceGroupName=` and `Guice > provision errors` due to `serviceGroup=null` in `ServiceFactory.createNew()`. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > 25cfaf84c56b9b84f806ce43b9f8cd3fa4d36265 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 1bf5299e3c687403cbb8ba72b670929de5f53a12 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostRequest.java > 2554c2544875f70bb10432dc4bd5685f0b3c9643 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ServiceGroupDAO.java > 97659f74b5df2d837068644bce8dbd3e5319da1c > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntity.java > 8994384956db0bac467cb61fd13afba07bbd21ab > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceGroupEntity.java > fedaee86fbbf4286b76a1c4758651f535bb6f642 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java > c479cdd42742ebada75f57efafd764d6b2112148 > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > bdda9a79ec952fda4ee64c46c6e2ed1c7c9e2d58 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > aff6d74f483ec365e3c8a96124f66e808c5b95ff > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > 14ab8cf7baa26771f3b9eb72cf62518f9e93a7ce > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > 0cc64e9d66b9afb21c62ab5b6346166d58e42036 > > ambari-server/src/test/java/org/apache/ambari/server/controller/BackgroundCustomCommandExecutionTest.java > 0b07fb7f1b31dde693116a97edc3a5425a82ba37 > > ambari-server/src/test/java/org/apache/ambari/server/controller/RefreshYarnCapacitySchedulerReleaseConfigTest.java > 99a7de370572ba055c31a056ea9012a66522 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AbstractResourceProviderTest.java > 0f04e744bad5f4005428dc018c9850aecdccc3d5 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ComponentResourceProviderTest.java > be7134e7c51426e941b92b3ac8aaa9f9935c0095 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentResourceProviderTest.java > bad63b789463653df5eaaabf89a2b0b917cebdbf > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java > d8575e7d6dfa9bda5671027bdf3ff152749a9ee9 > > ambari-s
Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review194031
---
Fix it, then Ship it!
I've [attached a small unit
test](https://issues.apache.org/jira/secure/attachment/12902568/AMBARI-22649-test.patch)
for `settings.py` to
[AMBARI-22649](https://issues.apache.org/jira/browse/AMBARI-22649), please feel
free to use it. I think it simplifies testing behavior of this library.
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 28 (patched)
<https://reviews.apache.org/r/64594/#comment272728>
Default value of `None` for `setting_names` would make it easier to get all
settings.
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 47-49 (patched)
<https://reviews.apache.org/r/64594/#comment272719>
Shouldn't this also return `None`? Now it says `/agentConfigParams` is not
supported, but goes on to return the requested values.
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 56 (patched)
<https://reviews.apache.org/r/64594/#comment272721>
Why is a `list`, `frozenset` or `tuple` not acceptable?
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 60 (patched)
<https://reviews.apache.org/r/64594/#comment272720>
This is unreachable, since `None` is not an instance of `set`, hence it
already returns `None` a bit earlier, instead of returning all settings here.
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 69 (patched)
<https://reviews.apache.org/r/64594/#comment272722>
I think if the specified keys are not present, instead of `None` it should
just return the empty `dict` that `result` already contains.
For example:
```
get_setting_type_entries('/clusterSettings', None)['key']
```
and
```
get_setting_type_entries('/clusterSettings', set(['key']))['key']
```
should have the same result (actual value or `KeyError`), but returning
`None` means the second one may result in `TypeError: 'NoneType' object has no
attribute '__getitem__'` instead.
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 89 (patched)
<https://reviews.apache.org/r/64594/#comment272727>
Should `return None` early if `setting_name is None`
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 105-108 (patched)
<https://reviews.apache.org/r/64594/#comment272726>
Could be simplified to
```
return setting_type in (STACK_SETTINGS_TYPE, CLUSTER_SETTINGS_TYPE)
```
- Attila Doroszlai
On Dec. 17, 2017, 9:11 a.m., Swapan Shridhar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
>
> (Updated Dec. 17, 2017, 9:11 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya,
> Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced
> "cluster settings" in Ambari.
>
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents
> in command*.json.**
> **==**
>
> One should be able to query for a given **clusterSettings** or
> **stackSettings**:
> - by passing in the setting name(one or more) in order to get it back as
> key-value map, or
> - just get the value back for a passed-in setting.
>
>
> **Functions for clusterSettings:**
> **--**
> - **get_cluster_setting_entries(setting_names)** :
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
>
> - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
>
> - **is_security_enabled()** :
> -- Retrieves the cluster's security status.
>
>
> **Functi
Review Request 64670: AMBARI-22660. Fix unit tests failing due to ServiceGroup issues
: 25, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.stack.StackManagerTest -Tests run: 10, Failures: 0, Errors: 10, Skipped: 0 - in org.apache.ambari.server.state.ServiceComponentTest +Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.ServiceComponentTest -Tests run: 6, Failures: 0, Errors: 6, Skipped: 0 - in org.apache.ambari.server.state.alerts.AlertEventPublisherTest +Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.alerts.AlertEventPublisherTest -Tests run: 17, Failures: 0, Errors: 17, Skipped: 0 - in org.apache.ambari.server.state.alerts.AlertReceivedListenerTest +Tests run: 17, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.alerts.AlertReceivedListenerTest -Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 - in org.apache.ambari.server.state.alerts.InitialAlertEventTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.alerts.InitialAlertEventTest -Tests run: 3, Failures: 0, Errors: 3, Skipped: 0 - in org.apache.ambari.server.state.cluster.AlertDataManagerTest +Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.cluster.AlertDataManagerTest -Tests run: 4, Failures: 0, Errors: 4, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClusterDeadlockTest +Tests run: 4, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClusterDeadlockTest -Tests run: 35, Failures: 3, Errors: 15, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClusterTest +Tests run: 35, Failures: 2, Errors: 7, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClusterTest -Tests run: 3, Failures: 0, Errors: 3, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClustersDeadlockTest +Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClustersDeadlockTest -Tests run: 9, Failures: 0, Errors: 3, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClustersTest +Tests run: 9, Failures: 0, Errors: 2, Skipped: 0 - in org.apache.ambari.server.state.cluster.ClustersTest -Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 - in org.apache.ambari.server.state.services.RetryUpgradeActionServiceTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 - in org.apache.ambari.server.state.services.RetryUpgradeActionServiceTest -Tests run: 13, Failures: 1, Errors: 2, Skipped: 2 - in org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest +Tests run: 13, Failures: 1, Errors: 1, Skipped: 2 - in org.apache.ambari.server.state.svccomphost.ServiceComponentHostTest ``` Thanks, Attila Doroszlai
Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64594/#review194013 --- ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py Lines 32-88 (patched) <https://reviews.apache.org/r/64594/#comment272704> Most of these 2 functions are the same as those in `cluster_settings`. Can you please try to reduce duplication? - Attila Doroszlai On Dec. 14, 2017, 10:05 p.m., Swapan Shridhar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64594/ > --- > > (Updated Dec. 14, 2017, 10:05 p.m.) > > > Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, > Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi. > > > Bugs: AMBARI-22649 > https://issues.apache.org/jira/browse/AMBARI-22649 > > > Repository: ambari > > > Description > --- > > Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced > "cluster settings" in Ambari. > > **==** > **Library for querying _clusterSettings_ and _stackSettings_ for its contents > in command*.json.** > **==** > > One should be able to query for a given **clusterSettings** or > **stackSettings**: > - by passing in the setting name(one or more) in order to get it back as > key-value map, or > - just get the value back for a passed-in setting. > > > **Functions for clusterSettings:** > **--** > - **get_cluster_setting_entries(setting_names)** : > -- Retrieves the passed-in cluster setting entr(y/ies) and their values > as a map. >If 'setting_names' is passed-in as None : all the settings names and > their corresponding values will be returned as map. >If 'setting_names' is passed-in as empty set : None will be returned. > > - **get_cluster_setting_value(setting_name)** : > -- Retrieves the passed-in cluster setting entry's value. > > - **is_security_enabled()** : > -- Retrieves the cluster's security status. > > > **Functions for stackSettings:** > **** > > Stack settings as of now has 5 settings : stack_name, stack_root, > stack_features, stack_tools, stack_packages. stack_name, stack_root have > string as values, whereas stack_features, stack_tools, stack_packages have > values as JSON. Further there already exists python functions in files : > **stack_features.py**, **stack_tools.py** and **stack_select.py**. > >- **get_stack_setting_entries(setting_names)** : > -- Retrieves the passed-in stack setting entr(y/ies) and their values > as a map. > If 'setting_names' is passed-in as None, all the settings names > and their corresponding values will be returned as map. > If 'setting_names' is passed-in as empty set : None will be > returned. > >- **get_stack_setting_value(setting_name)**: > -- Retrieves the passed-in stack setting entry's value. > > - **get_stack_name()**: > -- Retrieves the stack name. > > - **get_stack_root()**: >-- Retrieves the stack root. > > > > **Modifications in _stack_features.py, stack_tools.py and stack_select.py_ > files:** > **-** > > - Given that these already exist and as of now they read the relevant stack > setting from *configurations/cluster_env*. > - Thus, code has been added to try reading from /stackSettings first by > calling the new fn.() get_stack_setting_value(). if setting not found, go for > the fall back *configurations/cluster_env* (which would be removed soon, > when we remove cluster_env). > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py > PRE-CREATION > > ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py > 92823b0 > > ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py > b741a33 > > ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py > PRE-CREATION > > ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.p
Re: Review Request 64651: Revert adding repo_version column for stack table.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64651/#review193925 --- Ship it! Ship It! - Attila Doroszlai On Dec. 15, 2017, 4:47 p.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64651/ > --- > > (Updated Dec. 15, 2017, 4:47 p.m.) > > > Review request for Ambari, Attila Doroszlai, Jayush Luniya, Swapan Shridhar, > and Vitalyi Brodetskyi. > > > Bugs: AMBARI-22297 > https://issues.apache.org/jira/browse/AMBARI-22297 > > > Repository: ambari > > > Description > --- > > Stack table does not need the repo_version column, so it should be removed. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java > c479cdd427 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > dc77f19d63 > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 84349b6d2f > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql f9d5321e76 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql bc36e35a49 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 227645f1bb > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql > 333a0c3bc3 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 120bdb60d4 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java > 4313ae6028 > ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java > 1a8340ab0a > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java > d40679aea7 > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java > 730d51cb98 > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java > 558b334267 > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java > ca7a7aceb3 > > ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java > 8568b8c77e > > > Diff: https://reviews.apache.org/r/64651/diff/1/ > > > Testing > --- > > - Fixed related unit tests in ambari-server. > - Unit test run: slightly improved unit test results on the feature branch. > See below. > > Before patch test results: > > Tests run: 4946, Failures: 216, Errors: 449, Skipped: 35 > > After patch test results: > > Tests run: 4950, Failures: 216, Errors: 421, Skipped: 35 > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 64606: Fix unit tests in feature branch to make them workable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64606/#review193800 --- Ship it! Ship It! - Attila Doroszlai On Dec. 14, 2017, 2:46 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64606/ > --- > > (Updated Dec. 14, 2017, 2:46 p.m.) > > > Review request for Ambari, Attila Doroszlai and Dmytro Sen. > > > Bugs: AMBARI-22614 > https://issues.apache.org/jira/browse/AMBARI-22614 > > > Repository: ambari > > > Description > --- > > Fix unit tests in feature branch to make them workable > > > Diffs > - > > > ambari-server/src/test/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnableTest.java > 62ff3d3 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/DefaultRendererTest.java > 96f80b1 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java > 782f148 > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesUpCheckTest.java > c79e761 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > f2ee106 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariServerTest.java > 3e4baed > > ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java > 0f99b08 > > ambari-server/src/test/java/org/apache/ambari/server/controller/RefreshYarnCapacitySchedulerReleaseConfigTest.java > 3ce6686 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java > 7ccdadd > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java > 4b5d6c0 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > a6fc919 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ComponentResourceProviderTest.java > efa351a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentResourceProviderTest.java > 6611c55 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java > e23ad79 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java > 6bc8b95 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RootServiceHostComponentResourceProviderTest.java > ce3656c > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ServiceResourceProviderTest.java > 4593409 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackConfigurationResourceProviderTest.java > 91c048b > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java > 1bc8776 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java > a38e44d > > ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java > 9df236f > ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java > dafc5e8 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDispatchDAOTest.java > 4dac019 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java > 933743f > > ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java > 658ba84 > > ambari-server/src/test/java/org/apache/ambari/server/testing/DBInconsistencyTests.java > fafeaff > > ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java > 8546a9b > > > Diff: https://reviews.apache.org/r/64606/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Review Request 64614: AMBARI-22164. Fix wrong assignment due to leftover parameter name
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64614/ --- Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Madhuvanthi Radhakrishnan, and Swapan Shridhar. Bugs: AMBARI-22164 https://issues.apache.org/jira/browse/AMBARI-22164 Repository: ambari Description --- `serviceDisplayName` was changed to `serviceName` in both sides of the assignment, but the parameter name was left as was, resulting in the member variable `serviceName` always being set to `null`. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostRequest.java ea082c66e128a30368d6be195f3ee270a08e8518 Diff: https://reviews.apache.org/r/64614/diff/1/ Testing --- Tested in blueprint deployment. Thanks, Attila Doroszlai
Review Request 64613: AMBARI-22249. Fix AmbariContext.serviceGroupDependencyResourceProvider initialization
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64613/ --- Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Swapan Shridhar, and Vitalyi Brodetskyi. Bugs: AMBARI-22249 https://issues.apache.org/jira/browse/AMBARI-22249 Repository: ambari Description --- Wrong variable is initialized. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java f4137683aaad84d5a66b93b4559abd7a1fb926bc Diff: https://reviews.apache.org/r/64613/diff/1/ Testing --- Trivial fix. Thanks, Attila Doroszlai
Review Request 64611: AMBARI-22196. yum repo file contains parsing errors due to initial whitespace
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64611/ --- Review request for Ambari, Balázs Bence Sári, Jayush Luniya, and Swapan Shridhar. Bugs: AMBARI-22196 https://issues.apache.org/jira/browse/AMBARI-22196 Repository: ambari Description --- yum repo file created by Ambari results in `File contains parsing errors` due to initial whitespace Diffs - ambari-server/src/main/resources/cluster-settings.xml 70b100aba3d1378a7eb0d29c1c6dc609b42d8d89 Diff: https://reviews.apache.org/r/64611/diff/1/ Testing --- Tested package installation via blueprint deployment. Thanks, Attila Doroszlai
Review Request 64568: AMBARI-22638. yarn.timeline-service property enabled overriding preset secure blueprint config for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64568/ --- Review request for Ambari, Attila Magyar, Balázs Bence Sári, Robert Levas, and Robert Nettleton. Bugs: AMBARI-22638 https://issues.apache.org/jira/browse/AMBARI-22638 Repository: ambari Description --- Changed `yarn.timeline-service.enabled` in `kerberos.json` to `false`, to match stack default for HDP 3.0. Diffs - ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/kerberos.json a3cfaea95918e629f02d2fa9d6eb72a5c77e5987 Diff: https://reviews.apache.org/r/64568/diff/1/ Testing --- Tested secure cluster deployment with HDP 3.0, verified value of `yarn.timeline-service.enabled` property: `false` if unset or set to `false` in blueprint, `true` if set to `true` in blueprint. Unit tests: ``` $ mvn -am -pl ambari-server clean test -Dtest='kerberos/*' ... Tests run: 205, Failures: 0, Errors: 0, Skipped: 1 ... Total run:1210 Total errors:0 Total failures:0 ... BUILD SUCCESS ``` Thanks, Attila Doroszlai
Re: Review Request 64478: Blueprint V2 API: object model, resource provider, cluster creation
> On Dec. 11, 2017, 11:23 p.m., Madhuvanthi Radhakrishnan wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > > Line 1029 (original), 1023 (patched) > > <https://reviews.apache.org/r/64478/diff/1/?file=1911898#file1911898line1031> > > > > Hi Attila, > > Since we havent completely gotten rid of /cluster/configuration yet, > > the serviceId in the request can be null. The change you have made is what > > we will do eventually but right now this might break the existing feature > > branch deployments. Both code versions have the same result: * if `request.getServiceId()` is not `null`, then its value is used * if `request.getServiceId()` is `null`, then `null` is used, which is exactly `request.getServiceId()`'s value in this case The `?:` is unnecessary. > On Dec. 11, 2017, 11:23 p.m., Madhuvanthi Radhakrishnan wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseClusterRequest.java > > Line 137 (original), 110 (patched) > > <https://reviews.apache.org/r/64478/diff/1/?file=1911908#file1911908line150> > > > > Missing throws clause Throws `IllegalArgumentException` instead. Used in lambdas, hence cannot throw checked exception. Does not make much difference, because `InvalidTopologyTemplateException` is translated to `IllegalArgumentException` in `ClusterResourceProvider`, `BlueprintV2ResourceProvider` etc. anyway. > On Dec. 11, 2017, 11:23 p.m., Madhuvanthi Radhakrishnan wrote: > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java > > Line 66 (original), 66 (patched) > > <https://reviews.apache.org/r/64478/diff/1/?file=1911922#file1911922line66> > > > > If each service instance can have its own config properties then > > shouldnt it be serviceName (unique)? > > Correct me if i am wrong. It should be `serviceType`, because `AmbariMetaInfo.getServiceProperties` expects `stackServiceName` (which is the same). This line of change is not directly related to the blueprints implementation, it is an addendum for [AMBARI-22244](https://issues.apache.org/jira/browse/AMBARI-22244). > On Dec. 11, 2017, 11:23 p.m., Madhuvanthi Radhakrishnan wrote: > > ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java > > Line 194 (original), 200 (patched) > > <https://reviews.apache.org/r/64478/diff/1/?file=1911938#file1911938line208> > > > > Missing throws clause Same as above. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64478/#review193429 --- On Dec. 9, 2017, 1:19 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64478/ > --- > > (Updated Dec. 9, 2017, 1:19 p.m.) > > > Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Madhuvanthi > Radhakrishnan, Robert Nettleton, and Swapan Shridhar. > > > Bugs: AMBARI-22253, AMBARI-22297 and AMBARI-22325 > https://issues.apache.org/jira/browse/AMBARI-22253 > https://issues.apache.org/jira/browse/AMBARI-22297 > https://issues.apache.org/jira/browse/AMBARI-22325 > > > Repository: ambari > > > Description > --- > > Implementation of Blueprint V2 API POST that allows cluster creation. > Co-authored by Balázs Bence Sári and Attila Doroszlai. > > Includes minor fixes for: > > * AMBARI-22196 -- yum repo file contains parsing errors due to initial > whitespace > * AMBARI-22249 -- AmbariContext.serviceGroupDependencyResourceProvider > initialization > * AMBARI-22164 -- wrong assignment due to leftover parameter name > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/ObjectNotFoundException.java > 75c9f3b5d50102f3beeeaa8041178cc08e22e241 > > ambari-server/src/main/java/org/apache/ambari/server/ParentObjectNotFoundException.java > 15bd7cb6bbe5026bf0a5ae9d3fc0bec575137643 > > ambari-server/src/main/java/org/apache/ambari/server/StackAccessException.java > b8bfff3991a4e49514801dbb480958941cff01ac > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > fd43edfa88339cd94686ddc13822b36ab7c74d6f > > ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java > 2582c71eb95255f0f8081e90473183243f12cdd0 > > ambari-serve
Review Request 64478: Blueprint V2 API: object model, resource provider, cluster creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64478/ --- Review request for Ambari, Balázs Bence Sári, Jayush Luniya, Madhuvanthi Radhakrishnan, Robert Nettleton, and Swapan Shridhar. Bugs: AMBARI-22253, AMBARI-22297 and AMBARI-22325 https://issues.apache.org/jira/browse/AMBARI-22253 https://issues.apache.org/jira/browse/AMBARI-22297 https://issues.apache.org/jira/browse/AMBARI-22325 Repository: ambari Description --- Implementation of Blueprint V2 API POST that allows cluster creation. Co-authored by Balázs Bence Sári and Attila Doroszlai. Includes minor fixes for: * AMBARI-22196 -- yum repo file contains parsing errors due to initial whitespace * AMBARI-22249 -- AmbariContext.serviceGroupDependencyResourceProvider initialization * AMBARI-22164 -- wrong assignment due to leftover parameter name Diffs - ambari-server/src/main/java/org/apache/ambari/server/ObjectNotFoundException.java 75c9f3b5d50102f3beeeaa8041178cc08e22e241 ambari-server/src/main/java/org/apache/ambari/server/ParentObjectNotFoundException.java 15bd7cb6bbe5026bf0a5ae9d3fc0bec575137643 ambari-server/src/main/java/org/apache/ambari/server/StackAccessException.java b8bfff3991a4e49514801dbb480958941cff01ac ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java fd43edfa88339cd94686ddc13822b36ab7c74d6f ambari-server/src/main/java/org/apache/ambari/server/checks/ServiceCheckValidityCheck.java 2582c71eb95255f0f8081e90473183243f12cdd0 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 59e6622f708e7d8d27c9e28ce2ce58f014a4a5dc ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java c88b23c8af374dd9bd695b70560079871f3366fd ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationRequest.java d26cf16b59897573173ea258133a1c1b4e6f31f1 ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java f2ff181da0327a82adb67e1405ab9ae238b5c4d4 ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java cc84ce24b5e769257ac0e8744a93e3e236bb04e3 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java 6d0f3fbd4288d31c3e077445e20c3b2bac696930 ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostRequest.java ea082c66e128a30368d6be195f3ee270a08e8518 ambari-server/src/main/java/org/apache/ambari/server/controller/StackV2.java 0b0329c31bb5800eb32ba4b437bd57a41a618dbe ambari-server/src/main/java/org/apache/ambari/server/controller/StackV2Factory.java 784b3683ba9d3909bb9e6d900e4175b6a00aec9a ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 5104354fce7701fe023723c19d56d25e7adf5219 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseClusterRequest.java a38f478ed05ec88816e6363b31aa102a4a9d353f ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintV2ResourceProvider.java 6d2c4f06f5c75fbdf9e81c2d711440862f375bb9 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java cc0cea377233931bf65943f792f6a7e11b074f06 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java 565369b08b6eeb1a598575e448ce0c6334b6234b ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java f773fc8aac61f74988dd63a7a8b2cc30053023d2 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationResourceProvider.java 5a6dff4144bc059ff2614fb1ee4704c0c7a64809 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java 1284c265c150aa03c00b43b364c2163ca4e564a5 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 38e1233308a7f423789a1961e3dd8664919f29ae ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java cd23e834b160f889326f65f68162d5b28906e050 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java 2f0974e95d7dc27c9b3e33fa426b0e5dc3b3922c ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java c479cdd42742ebada75f57efafd764d6b2112148 ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 6bb248e19bef39ba2edb3671d38ef815d8efb593 ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java dc77f19d6389071e45fccb31942c12c8f49e59e0 ambari-server/src/main/java/org/apache/ambari/server/state
Re: Review Request 64397: AMBARI-22602. Add 'clusterSettings' and 'stackSettings' parameters in Execution Command.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64397/#review193274
---
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
Lines 436 (patched)
<https://reviews.apache.org/r/64397/#comment271827>
* no need for explicit type arg in Maps.newHashMap()
* empty map should be created lazily in `else` below
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
Lines 438 (patched)
<https://reviews.apache.org/r/64397/#comment271831>
According to IDEA there is no need to include the `clusterSettings.values()
!= null` check, as it is always true if `clusterSettings != null` is true.
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
Lines 449-451 (patched)
<https://reviews.apache.org/r/64397/#comment271828>
Please use `collect(toMap())` (as above) instead of `forEach` and `put`
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
Lines 2676-2695 (patched)
<https://reviews.apache.org/r/64397/#comment271829>
Can you please extract the logic to 2 utility methods to avoid code
duplication among `AmbariCustomCommandExecutionHelper`,
`AmbariManagementControllerImpl` and `ClientConfigResourceProvider`?
- Attila Doroszlai
On Dec. 7, 2017, 9:11 a.m., Swapan Shridhar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64397/
> ---
>
> (Updated Dec. 7, 2017, 9:11 a.m.)
>
>
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
>
>
> Bugs: AMBARI-22602
> https://issues.apache.org/jira/browse/AMBARI-22602
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-22198 added "stack settings", and AMBARI-22196 introduced "cluster
> settings" in Ambari.
>
> This review adds 2 new parameters to Execution Command : **(1).**
> clusterSettings and **(2).** stackSettings, and adds their respective set of
> key-values pairs.
>
> - This enables these parameters to be passed in to the ambari-agent's
> **command*.json**.
>
>
> Test cases are tracked in : AMBARI-22603
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
> 5ee4bf6
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
> e7dea06
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
> 59e6622
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
> 9043297
>
> ambari-server/src/main/java/org/apache/ambari/server/resources/RootLevelSettingsManager.java
> 8bd24f2
>
>
> Diff: https://reviews.apache.org/r/64397/diff/1/
>
>
> Testing
> ---
>
> Tested on Live cluster doing following :
> (1). Start Service
> (2). Restart Service
> (3). Stop Service
> (4). Refreshing client configs
> (5). Adding Client.
>
>
> **Snippet from command*.json as part of testing:**
> {
> ...
> ...
> "clusterSettings": {
> "security_enabled": "false",
> "hide_yarn_memory_widget": "false",
> "enable_external_ranger": "false",
> "override_uid": "true",
> "kerberos_domain": "EXAMPLE.COM",
> "one_dir_per_partition": "false",
> "repo_ubuntu_template": "{{package_type}} {{base_url}}
> {{components}}",
> "ignore_groupsusers_create": "false",
> "alerts_repeat_tolerance": "1",
> "namenode_rolling_restart_timeout": "4200",
> "fetch_nonlocal_groups": "true",
> "manage_dirs_on_root": "true",
> "recovery_lifetime_max_count": "1024",
> "agent_mounts_ignore_list": "",
> "ignore_bad_mounts": "false",
> "recovery_window_in_minutes": "60",
> "sysprep_skip_copy_tarballs_hdfs": "false",
> "recovery_type": "AUTO_START"
Re: Review Request 64420: Fix unit tests in feature branch to make them workable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64420/#review193254 --- Ship it! Ship It! - Attila Doroszlai On Dec. 7, 2017, 5:59 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64420/ > --- > > (Updated Dec. 7, 2017, 5:59 p.m.) > > > Review request for Ambari, Attila Doroszlai and Jayush Luniya. > > > Bugs: AMBARI-22614 > https://issues.apache.org/jira/browse/AMBARI-22614 > > > Repository: ambari > > > Description > --- > > Fix unit tests in feature branch to make them workable > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > fc49a24 > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java > 69eb21d > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > 0b38b36 > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java > 19f940f > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionSchedulerThreading.java > 22f23c8 > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java > 8a6025f > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java > 50d4733 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java > 4f8c423 > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/ClusterResourceDefinitionTest.java > fc24d2c > > ambari-server/src/test/java/org/apache/ambari/server/api/resources/ServiceResourceDefinitionTest.java > 75319ea > > ambari-server/src/test/java/org/apache/ambari/server/checks/ComponentExistsInRepoCheckTest.java > 2fe8ee0 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java > 040b571 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > 21cf518 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > 5c6f374 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProviderTest.java > aee1952 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java > 9e6b98b > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java > 387fa45 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java > 8e968cc > > ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/state/GeneralServiceCalculatedStateTest.java > 0b6c3f3 > > ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java > a86d125 > ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java > a03d7cc > > ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java > 158c47d > ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java > 4e1c139 > > ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java > 5e5395a > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java > a52f438 > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java > 4ce7387 > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java > be4a907 > ambari-server/src/test/resources/cluster-settings.xml PRE-CREATION > > > Diff: https://reviews.apache.org/r/64420/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 64206: Upgrade Ambari Infra to use Solr 5.5.5 for Ambari 2.6.1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64206/#review192278 --- Ship it! Ship It! - Attila Doroszlai On Nov. 30, 2017, 12:51 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64206/ > --- > > (Updated Nov. 30, 2017, 12:51 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Krisztian Kasa, > and Miklos Gergely. > > > Bugs: AMBARI-22559 > https://issues.apache.org/jira/browse/AMBARI-22559 > > > Repository: ambari > > > Description > --- > > upgrade solr version (also use horton repo as default) > > > Diffs > - > > ambari-logsearch/ambari-logsearch-assembly/pom.xml c0a92f4 > ambari-logsearch/pom.xml 5a1edee > > > Diff: https://reviews.apache.org/r/64206/diff/1/ > > > Testing > --- > > not required > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64173: AMBARI-22550. Service install/start failure due to wrong version directory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64173/ --- (Updated Nov. 29, 2017, 9:21 p.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Nettleton. Changes --- doc Bugs: AMBARI-22550 https://issues.apache.org/jira/browse/AMBARI-22550 Repository: ambari Description --- Only send `/commandParams/version` if it is resolved: using unresolved version eg. `2.6.3.0` only causes problems. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java 212363b85f1ef4f4b3b91f0ddb2fe91cd897d5dc ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java 7630c553c0e55ffa314f02bc9c68337d502a93be Diff: https://reviews.apache.org/r/64173/diff/2/ Changes: https://reviews.apache.org/r/64173/diff/1-2/ Testing --- * Tested cluster creation: verified that resolved version (eg. 2.6.3.0-235) is sent in "start" commands (and version is absent in "install" commands), and that WebHCat server is started OK. * Tested service auto-restart for regression per steps in [AMBARI-22025](https://issues.apache.org/jira/browse/AMBARI-22025). * Added unit test. `Tests run: 4931, Failures: 0, Errors: 0, Skipped: 34` Thanks, Attila Doroszlai
Review Request 64173: AMBARI-22550. Service install/start failure due to wrong version directory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64173/ --- Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Nettleton. Bugs: AMBARI-22550 https://issues.apache.org/jira/browse/AMBARI-22550 Repository: ambari Description --- Only send `/commandParams/version` if it is resolved: using unresolved version eg. `2.6.3.0` only causes problems. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java 212363b85f1ef4f4b3b91f0ddb2fe91cd897d5dc ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java 7630c553c0e55ffa314f02bc9c68337d502a93be Diff: https://reviews.apache.org/r/64173/diff/1/ Testing --- * Tested cluster creation: verified that resolved version (eg. 2.6.3.0-235) is sent in "start" commands (and version is absent in "install" commands), and that WebHCat server is started OK. * Tested service auto-restart for regression per steps in [AMBARI-22025](https://issues.apache.org/jira/browse/AMBARI-22025). * Added unit test. `Tests run: 4931, Failures: 0, Errors: 0, Skipped: 34` Thanks, Attila Doroszlai
Re: Review Request 64153: Oozie stop failed during IOP migration with missing module get_lzo_packages
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64153/#review192116 --- Ship it! Ship It! - Attila Doroszlai On Nov. 29, 2017, 1:22 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64153/ > --- > > (Updated Nov. 29, 2017, 1:22 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Sid > Wagle, and Vitalyi Brodetskyi. > > > Bugs: AMBARI-22541 > https://issues.apache.org/jira/browse/AMBARI-22541 > > > Repository: ambari > > > Description > --- > > **STR** > > 1. Deployed cluster with Ambari version: 2.4.2.0_IBM_00 and HDP > version: 4.2.5.0- (LZO is enabled in HDFS configs) > 2. Upgrade Ambari to Target Version: 2.6.1.0-64 | Hash: > cd4db8e9ac0ea7ce14fc1253959a121688f34952 > 3. Register HDP-2.6.4.0-51 version and call remove iop-slect > 4. Install the bits and start EU > > **Result** > EU failed at Stop Oozie server > > > > > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py", > line 148, in > OozieServer().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 367, in execute > method(env) > File > "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py", > line 84, in stop > import params > File > "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py", > line 28, in > from params_linux import * > File > "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py", > line 33, in > from resource_management.libraries.functions.get_lzo_packages import > get_lzo_packages > ImportError: No module named get_lzo_packages > > > Live cluster: <http://172.22.87.3:8080/#/main/admin/stack/upgrade> > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py > 33a309fa2e > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py > 0e6179b8a7 > > > Diff: https://reviews.apache.org/r/64153/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Review Request 64147: AMBARI-22539. Cannot add host with host_name in request body
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64147/ --- Review request for Ambari, Balázs Bence Sári, Nate Cole, and Robert Nettleton. Bugs: AMBARI-22539 https://issues.apache.org/jira/browse/AMBARI-22539 Repository: ambari Description --- Support for one kind of `host_name` specification was [inadvertently removed](https://git1-us-west.apache.org/repos/asf?p=ambari.git;a=blobdiff;f=ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java;h=f9604900af1e84756d2aeed3e6cee864b3586c58;hp=2a91bfe07536a4a0f49497f24aafa9108e257f94;hb=48f7fb22ae2ea61d7b05fa24703ccff2941bf8ff;hpb=88d93b92429225e03f46cb739a738b9033d45b84) from `ScaleClusterRequest` in Ambari 2.6.0. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java 0df8f13bd922547b5951ad9bc70dd83524c5b442 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ScaleClusterRequest.java ad90a2559b8a644b1855fedca3c6a55dc663f20f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ScaleClusterRequestTest.java b8395570448b75e74cf075110467a4279051f9ac Diff: https://reviews.apache.org/r/64147/diff/1/ Testing --- Tested manually per STR. Added unit test. Thanks, Attila Doroszlai
Re: Review Request 64083: Log Search: Adding new dynamic field for service logs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64083/#review191890 --- Ship it! Ship It! - Attila Doroszlai On Nov. 27, 2017, 4:13 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64083/ > --- > > (Updated Nov. 27, 2017, 4:13 p.m.) > > > Review request for Ambari, Attila Doroszlai, Krisztian Kasa, and Robert > Nettleton. > > > Bugs: Ambari > https://issues.apache.org/jira/browse/Ambari > > > Repository: ambari > > > Description > --- > > its required to add in order to have a dynamic field which not only indexed, > but stored as well. > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-server/src/main/configsets/hadoop_logs/conf/managed-schema > c6f498b > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/solr/SolrConstants.java > 60fc1a3 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/solr/model/SolrServiceLogData.java > c6fdba3 > > > Diff: https://reviews.apache.org/r/64083/diff/3/ > > > Testing > --- > > done. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64027: Ambari server becomes unusable when config properties are misconfigured
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64027/ --- (Updated Nov. 22, 2017, 1:52 p.m.) Review request for Ambari, Attila Magyar, Balázs Bence Sári, and Miklos Gergely. Bugs: AMBARI-22499 https://issues.apache.org/jira/browse/AMBARI-22499 Repository: ambari Description --- Not all numeric values are valid doubles, eg. 86400L is a valid long, but not a double. Changed to `NumberUtils.createNumber` instead of `Double.parseDouble` to create the appropriate numeric object. Checking `doubleValue()` is necessary for equal values of different types (eg. `0.0` vs `0`). Also fixed possible NPE in non-numeric case. Diffs - ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java eade91458847125df8e50505cb2af5bd67ee0b3d ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java 8a0a78228e72fc14ff361445363fa61d1cc4e977 Diff: https://reviews.apache.org/r/64027/diff/1/ Testing --- Added unit test: ``` [INFO] Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.31 s - in org.apache.ambari.server.state.ConfigHelperTest ``` Thanks, Attila Doroszlai
Review Request 64027: Ambari server becomes unusable when config properties are misconfigured
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64027/ --- Review request for Ambari, Attila Magyar, Balázs Bence Sári, and Miklos Gergely. Repository: ambari Description --- Not all numeric values are valid doubles, eg. 86400L is a valid long, but not a double. Changed to `NumberUtils.createNumber` instead of `Double.parseDouble` to create the appropriate numeric object. Checking `doubleValue()` is necessary for equal values of different types (eg. `0.0` vs `0`). Also fixed possible NPE in non-numeric case. Diffs - ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java eade91458847125df8e50505cb2af5bd67ee0b3d ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java 8a0a78228e72fc14ff361445363fa61d1cc4e977 Diff: https://reviews.apache.org/r/64027/diff/1/ Testing --- Added unit test: ``` [INFO] Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.31 s - in org.apache.ambari.server.state.ConfigHelperTest ``` Thanks, Attila Doroszlai
Re: Review Request 63965: New endpoint service_config_version for creating new config versions via service endpoint instead of cluster update
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63965/#review191593 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java Lines 6081 (patched) <https://reviews.apache.org/r/63965/#comment269435> missing `<>` ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java Lines 6097-6098 (patched) <https://reviews.apache.org/r/63965/#comment269437> What's the use of calling these methods and ignoring the return values? ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java Lines 6135-6137 (patched) <https://reviews.apache.org/r/63965/#comment269440> What's the use of this `map`? I don't see any references later. - Attila Doroszlai On Nov. 20, 2017, 11:08 p.m., Madhuvanthi Radhakrishnan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63965/ > --- > > (Updated Nov. 20, 2017, 11:08 p.m.) > > > Review request for Ambari, Jayush Luniya, Swapan Shridhar, and Vitalyi > Brodetskyi. > > > Bugs: AMBARI-22487 > https://issues.apache.org/jira/browse/AMBARI-22487 > > > Repository: ambari > > > Description > --- > > Create new service config version > POST > api/v2/clusters//servicegroups//services//configurations/service_config_versions > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ConfigurationService.java > 79128a2374 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceConfigVersionService.java > c6e320121f > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > b5bc813e9c > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 875d9b6f9f > > ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceConfigVersionRequest.java > 29a7a421cd > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceConfigVersionResourceProvider.java > cbc08ebdf4 > > > Diff: https://reviews.apache.org/r/63965/diff/1/ > > > Testing > --- > > mvn clean install -DskipTests=true -Dmaven.test.skip=true -X > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main SUCCESS [ 20.681 > s] > [INFO] Apache Ambari Project POM .. SUCCESS [ 0.047 > s] > [INFO] Ambari Web . SUCCESS [ 59.888 > s] > [INFO] Ambari Views ... SUCCESS [ 1.454 > s] > [INFO] Ambari Admin View .. SUCCESS [ 9.794 > s] > [INFO] ambari-utility . SUCCESS [ 4.461 > s] > [INFO] ambari-metrics . SUCCESS [ 1.585 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 5.107 > s] > [INFO] Ambari Metrics Hadoop Sink . SUCCESS [ 3.697 > s] > [INFO] Ambari Metrics Flume Sink .. SUCCESS [ 3.057 > s] > [INFO] Ambari Metrics Kafka Sink .. SUCCESS [ 3.088 > s] > [INFO] Ambari Metrics Storm Sink .. SUCCESS [ 2.462 > s] > [INFO] Ambari Metrics Storm Sink (Legacy) . SUCCESS [ 2.196 > s] > [INFO] Ambari Metrics Collector ... SUCCESS [ 20.644 > s] > [INFO] Ambari Metrics Monitor . SUCCESS [ 2.597 > s] > [INFO] Ambari Metrics Grafana . SUCCESS [ 5.799 > s] > [INFO] Ambari Metrics Host Aggregator . SUCCESS [ 5.548 > s] > [INFO] Ambari Metrics Assembly SUCCESS [01:48 > min] > [INFO] Ambari Service Advisor . SUCCESS [ 0.614 > s] > [INFO] Ambari Server .. SUCCESS [03:49 > min] > [INFO] Ambari Functional Tests SUCCESS [ 0.494 > s] > [INFO] Ambari Agent ... SUCCESS [ 50.352 > s] > [INFO] Ambari Client .. SUCCESS [ 0.100 > s] > [INFO] Ambari Python Client ... SUCCESS [ 0.896
Re: Review Request 63838: ambari-server setup should surface GPL software agreement
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63838/#review191067 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java Lines 734-735 (patched) <https://reviews.apache.org/r/63838/#comment268661> I'd prefer type `Boolean` (and possibly default value of `Boolean.FALSE`). ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java Lines 5332 (patched) <https://reviews.apache.org/r/63838/#comment268659> Looks like this would throw NPE if the property `gpl.license.accepted` is not present in `ambari.properties`. How about using `Boolean.valueOf(String)`? - Attila Doroszlai On Nov. 15, 2017, 3:34 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63838/ > --- > > (Updated Nov. 15, 2017, 3:34 p.m.) > > > Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle. > > > Bugs: AMBARI-22453 > https://issues.apache.org/jira/browse/AMBARI-22453 > > > Repository: ambari > > > Description > --- > > * Ask the user if they are okay with installing GPL license software > * Surface the license agreement subtext, for now use placeholder > * Save the result in ambari.properties > * The user selection should be available through API for the client and > also stack scripts > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java > 91df506478 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 0bd6af6611 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > d8712f2c75 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > 0b048ad089 > ambari-server/src/main/python/ambari-server.py 60c60052cc > ambari-server/src/main/python/ambari_server/serverConfiguration.py > c7ac13476f > ambari-server/src/main/python/ambari_server/serverSetup.py c1643f2621 > > > Diff: https://reviews.apache.org/r/63838/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63832: Reference Log Feeder IDE/maven dev setup in README.md
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63832/#review191048 --- Ship it! Ship It! - Attila Doroszlai On Nov. 15, 2017, 12:35 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63832/ > --- > > (Updated Nov. 15, 2017, 12:35 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Krisztian Kasa, > and Robert Nettleton. > > > Bugs: AMBARI-22450 > https://issues.apache.org/jira/browse/AMBARI-22450 > > > Repository: ambari > > > Description > --- > > small patch to reference logfeeder reamdme in the parent readme > > > Diffs > - > > ambari-logsearch/README.md 33952a2 > > > Diff: https://reviews.apache.org/r/63832/diff/1/ > > > Testing > --- > > not required. > > > Thanks, > > Oliver Szabo > >
Review Request 63754: AMBARI-22426. Push down Logger instances
/ambari/server/controller/utilities/state/HBaseServiceCalculatedState.java eac0dce1e02c46803b056943fd1fe446762590b8 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/HDFSServiceCalculatedState.java 7bbad2ad5af11eb24ca0ba15c1efdee97ad67440 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/HiveServiceCalculatedState.java 77e44a5029bd04bf3042369583dccefcbeb15151 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/OozieServiceCalculatedState.java 1803f7041a3c5a71bdf7f8a690b9539466fed18d ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/YARNServiceCalculatedState.java 2f1619f27fd28b317e013d7f3e123997eac881c8 ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/DefaultLdapConnectionConfigService.java 9afcf51a4968c4740cac10b10a9fc94d0405156b ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RemoteAmbariClusterEntity.java 4f10d6f4247ef99116896f27b62460cfb97243c6 ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java b7ff297ce5eaa9042f4b6c5d8db57ffe509f6141 ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialProvider.java 1f2f6dbe8bf48d9f34fc3675766095cdf7efb765 ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KDCKerberosOperationHandler.java e5696cd48e862eb1e26eab279c1d4645de71cc52 ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java 510e7067f7816cbd93623cba80eec6bcb328c11f ambari-server/src/main/java/org/apache/ambari/server/state/stack/OsFamily.java c0483e839a47ff87964cf4fa35318b56c9d8d94a ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java 7bd377f183ca265026fbe138996607b02a8dc55f ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java 12af1310ecab35d18b4a0705229201e8d716ba1b ambari-server/src/main/java/org/apache/ambari/server/utils/RetryHelper.java d732edf4c7a41fd0ef762a1c4f04ae9b97d064ba ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationUtility.java 55f85a7e8ef53324e0091a528117a7a01cf45013 ambari-server/src/main/java/org/apache/ambari/server/view/ViewExtractor.java ad1cc5227976f3702bc852d264f4f2e79f58242f ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java c50276e8561182514dfc59e2a6901c6f476483ed ambari-server/src/main/java/org/apache/ambari/server/view/ViewSubResourceProvider.java 5cceb1ece5cb9221ee93c6530e30b1bcc945e803 ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 5d4c7581c4c8a7b22e527fac31bf7f54ec8212aa Diff: https://reviews.apache.org/r/63754/diff/1/ Testing --- Tested some requests that previously referred to "abstract" or "base" classes, to verify that the actual logging class shows up in the message, eg.: ``` 13 Nov 2017 14:19:14,137 ERROR [ambari-client-thread-37] CreateHandler:79 - Bad request received: Request body for Blueprint create request is empty 13 Nov 2017 14:19:14,133 ERROR [ambari-client-thread-37] BlueprintResourceProvider:194 - Exception while creating blueprint ``` Code compiles, unit tests pass: ``` $ mvn -am -pl ambari-server clean test ... Tests run: 4955, Failures: 0, Errors: 0, Skipped: 36 ... ``` Thanks, Attila Doroszlai
Re: Review Request 63725: Fix tar_archive.py: 1. remove verbosity flags, 2. archive_dir() and archive_directory_dereference() should store relative paths
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63725/#review190727 --- Ship it! Ship It! - Attila Doroszlai On Nov. 10, 2017, 4:26 p.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63725/ > --- > > (Updated Nov. 10, 2017, 4:26 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Andrew Onischuk, > and Jonathan Hurley. > > > Bugs: AMBARI-22413 > https://issues.apache.org/jira/browse/AMBARI-22413 > > > Repository: ambari > > > Description > --- > > Removed verbosity flag and fixed the two methods to store relative paths. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/tar_archive.py > e6d8924 > ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py > 8c48347 > ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 3114fa8 > > > Diff: https://reviews.apache.org/r/63725/diff/1/ > > > Testing > --- > > - Manually tested archiving / restoration at Falcon server start / stop. > - Manually tested HDP upgrades with Falcon server and BigInsight 4.2.5 -> HDP > upgrade (didn't not test the archival process only that upgrade went smoothly) > - Updates python unit tests. All python unit tests pass in ambari-server > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 63573: AMBARI-22366. POST, GET and UPDATE API for cluster settings. (/clusters/{clusterName}/settings).
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63573/#review190640
---
Ship it!
Ship It!
- Attila Doroszlai
On Nov. 9, 2017, 7:54 p.m., Swapan Shridhar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63573/
> ---
>
> (Updated Nov. 9, 2017, 7:54 p.m.)
>
>
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
>
>
> Bugs: AMBARI-22366
> https://issues.apache.org/jira/browse/AMBARI-22366
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-22366. POST, GET and UPDATE API for cluster settings.
> (/clusters/{clusterName}/settings).
>
> **Background:** AMBARI-22196 did the following :
> (1). Refactoring work of moving all cluster related configs/settings in
> cluster-env to "cluster-settings" files.
> (2). Implementing READ only API to access *cluster-settings* file
> *setting(s)* : *http://:/api/v1/cluster_settings/ *
>
> - This was done to separate cluster related configs from stack related
> configs.
>
> **This JIRA implements the following :**
>
> - POST, GET, UPDATE and DELETE APIs for cluster settings with endpoint :
> *http://:/api/v1/clusters/[[clusterName]]/settings*, where
>-- the cluster settings would be initially read from the above mentioned
> read-only API : *http://:/api/v1/cluster_settings/*.
>
> - Further, added cluster *settings* as a sub-resource of *cluster* resource.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/ClusterSettingNotFoundException.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
> e5680dd
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterSettingResourceDefinition.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
> 6096fa5
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
> 92f50f4
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterSettingService.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
> 358b1bf
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
> 370f735
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ClusterSettingRequest.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ClusterSettingResponse.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
> 3db55d4
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
> 20f4864
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceGroupResponse.java
> 147650c
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
> 9050e3d
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterSettingResourceProvider.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultProviderModule.java
> c219d23
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceGroupResourceProvider.java
> 2e935af
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
> bd6d7bb
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ClusterSettingEvent.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ClusterSettingDAO.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterEntity.java
> abbf709
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterSettingEntity.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/resources/RootLevelSettingsManager.java
> 3b74e69
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
> 5ba61b7
>
> ambari-server/src/main/java/org/apache/ambari/server/state/ClusterSetting.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/
Re: Review Request 63464: Beeline connection hangs for longer time when connection to HS2 with metastore DB down
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63464/#review190596 --- Doesn't the same problem affect Spark/Spark2 Thrift Server alert, which also uses Beeline? - Attila Doroszlai On Nov. 1, 2017, 1:43 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63464/ > --- > > (Updated Nov. 1, 2017, 1:43 p.m.) > > > Review request for Ambari and Vitalyi Brodetskyi. > > > Bugs: AMBARI-22346 > https://issues.apache.org/jira/browse/AMBARI-22346 > > > Repository: ambari > > > Description > --- > > PROBLEM: > beeline connection to HS2 hangs when hive metastore Database is down. > Issue seen on HDP 2.5.3 and 2.6 version clusters, same test on HDP 2.4.2 > beeline doesn't hangs. > > BUSINESS IMPACT: > Ambari spawning multiple beeline processes thats hanging causes unnecessary > memory usage, resulting in master service failures. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py > 3560bf82f5 > > > Diff: https://reviews.apache.org/r/63464/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63702: PERF stack: Deploy fails due to common hooks being used instead of one specific to PERF
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63702/#review190584 --- Ship it! Ship It! - Attila Doroszlai On Nov. 9, 2017, 3:14 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63702/ > --- > > (Updated Nov. 9, 2017, 3:14 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22402 > https://issues.apache.org/jira/browse/AMBARI-22402 > > > Repository: ambari > > > Description > --- > > Hooks location should be customizible. Because perf stack uses different hooks > folder. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java > 1bf67c3c24 > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java > aba4952603 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 2fd22d563a > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > a1ec98afe7 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > eb6737adb2 > contrib/utils/perf/deploy-gce-perf-cluster.py 7431ae9d7a > > > Diff: https://reviews.apache.org/r/63702/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63696: Exclude test scope jars from Log Search / Log Feeder rpm/deb packages
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63696/#review190575 --- Ship it! Ship It! - Attila Doroszlai On Nov. 9, 2017, 1:53 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63696/ > --- > > (Updated Nov. 9, 2017, 1:53 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Aravindan > Vijayan, Krisztian Kasa, Miklos Gergely, and Robert Nettleton. > > > Bugs: branch-2.6 and trunk > https://issues.apache.org/jira/browse/branch-2.6 > https://issues.apache.org/jira/browse/trunk > > > Repository: ambari > > > Description > --- > > dependency plugin include all jars in target/libs, so we need to remove those > that has test scope > > > Diffs > - > > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml cf64a36 > ambari-logsearch/ambari-logsearch-portal/pom.xml 53d6e8b > > > Diff: https://reviews.apache.org/r/63696/diff/2/ > > > Testing > --- > > UT done, target/libs folder does not contain easymock, checkstyle, > utility,junit jars > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63573: AMBARI-22366. POST, GET and UPDATE API for cluster settings. (/clusters/{clusterName}/settings).
t; -- where?
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
Lines 985-989 (patched)
<https://reviews.apache.org/r/63573/#comment267820>
Use `{}` placeholders instead of string concatenation, then
`isDebugEnabled` check is not needed
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
Lines 1009-1012 (patched)
<https://reviews.apache.org/r/63573/#comment267821>
Use `{}` placeholders instead of string concatenation, then
`isDebugEnabled` check is not needed
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
Lines 1126 (patched)
<https://reviews.apache.org/r/63573/#comment267819>
Explicit type parameter is unnecessary for `new HashMap<...>`
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
Lines 1653 (patched)
<https://reviews.apache.org/r/63573/#comment267818>
Seems to be left-over reference
- Attila Doroszlai
On Nov. 8, 2017, 12:59 a.m., Swapan Shridhar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63573/
> ---
>
> (Updated Nov. 8, 2017, 12:59 a.m.)
>
>
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
>
>
> Bugs: AMBARI-22366
> https://issues.apache.org/jira/browse/AMBARI-22366
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-22366. POST, GET and UPDATE API for cluster settings.
> (/clusters/{clusterName}/settings).
>
> **Background:** AMBARI-22196 did the following :
> (1). Refactoring work of moving all cluster related configs/settings in
> cluster-env to "cluster-settings" files.
> (2). Implementing READ only API to access *cluster-settings* file
> *setting(s)* : *http://:/api/v1/cluster_settings/ *
>
> - This was done to separate cluster related configs from stack related
> configs.
>
> **This JIRA implements the following :**
>
> - POST, GET, UPDATE and DELETE APIs for cluster settings with endpoint :
> *http://:/api/v1/clusters/[[clusterName]]/settings*, where
>-- the cluster settings would be initially read from the above mentioned
> read-only API : *http://:/api/v1/cluster_settings/*.
>
> - Further, added cluster *settings* as a sub-resource of *cluster* resource.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/ClusterSettingNotFoundException.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
> e5680dd
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterSettingResourceDefinition.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
> 6096fa5
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
> 92f50f4
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterSettingService.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
> 358b1bf
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
> 370f735
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ClusterSettingRequest.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ClusterSettingResponse.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
> 3db55d4
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
> 20f4864
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceGroupResponse.java
> 147650c
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
> 9050e3d
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterSettingResourceProvider.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultProviderModule.java
> c219d23
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceGroupResourceProvider.java
> 2e935af
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
> bd6d7bb
>
> ambari-server/src/main/java/org/apache/ambari/server/events/ClusterSettingEve
Re: Review Request 63618: Multiple issue with handling unexcepted situations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63618/#review190309 --- Fix it, then Ship it! ambari-agent/src/main/python/ambari_agent/ClusterCache.py Lines 78 (patched) <https://reviews.apache.org/r/63618/#comment267595> The cache still may end up in a partially updated state, since this does not clear it, just a local variable. ambari-agent/src/main/python/ambari_agent/ClusterCache.py Lines 99-100 (original), 103-106 (patched) <https://reviews.apache.org/r/63618/#comment267598> `persist_cache()` will write possibly obsolete `self.hash`, not the new one. Should pass `cache_hash` as argument now that the order is reversed. - Attila Doroszlai On Nov. 7, 2017, 2:48 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63618/ > --- > > (Updated Nov. 7, 2017, 2:48 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22375 > https://issues.apache.org/jira/browse/AMBARI-22375 > > > Repository: ambari > > > Description > --- > > * If on_update_cache fails cache should be discard > * loggers should log messages during initialization of classes > * hash of cache should only updated if saving/processing it was sucessful > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/ClusterCache.py b924420f8b > ambari-agent/src/main/python/ambari_agent/InitializerModule.py 2c8021844f > ambari-agent/src/main/python/ambari_agent/main.py 3948290519 > ambari-agent/src/test/python/ambari_agent/TestAgentStompResponses.py > 2154fca8d3 > > > Diff: https://reviews.apache.org/r/63618/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63357: AMBARI-22322. Alert definition support for multi-service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63357/ --- (Updated Nov. 6, 2017, 11:36 a.m.) Review request for Ambari, Jayush Luniya, Madhuvanthi Radhakrishnan, and Swapan Shridhar. Changes --- rebased Bugs: AMBARI-22322 https://issues.apache.org/jira/browse/AMBARI-22322 Repository: ambari Description --- * Introduce events for service type addition/removal to signal "first/last service of the same type" events. * Bind alert definition lifecycle to service type events instead of service events. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java c88c5145d0aa415fb78210084a1acb0a5c169ba7 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceGroupInstalledEvent.java e66d8a002a32336c49fc519a1ffe0676cf75ac12 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceGroupRemovedEvent.java b29a9a4fdc80d2d0bf0f61b10bed19d5483a34c0 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceInstalledEvent.java aa0a5ea8c30594208c574049cd98cd967db25a99 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceTypeAddedEvent.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/events/ServiceTypeEvent.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/events/ServiceTypeRemovedEvent.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertServiceStateListener.java 41ae4e8e9ceab1e1f009b5c44b995b9357f8fe6c ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 5c6a82ffcfa53dfc1dee6fd97e9098c4f1ce56d2 ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 27d37e3c43d572c471a29fa597a93e5f3e45fafe ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 025015a3e2bf30d5dfb714a46f3c1998a7183c6a ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 755a80fcc3fead63481b3df9acda858e6235dc64 ambari-server/src/main/java/org/apache/ambari/server/utils/EventBusSynchronizer.java c72ffbe032445fec725db04836d373c5ea15c603 ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 6ebd444e5541d84043ea8e51d1a2e50eb458b0ee ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java bd8be3b67225faa2bb33bedbd9fa42cb27dee694 Diff: https://reviews.apache.org/r/63357/diff/2/ Changes: https://reviews.apache.org/r/63357/diff/1-2/ Testing --- * Tested multiple service creation/removal manually * Adjusted `EventsTest` to cover the multiple services scenario Thanks, Attila Doroszlai
Re: Review Request 63360: Log Search: Add docker compose file for Solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63360/#review189455 --- ambari-logsearch/docker/solr.yml Lines 1 (patched) <https://reviews.apache.org/r/63360/#comment266554> Please add licence headers or add to exclude in pom.xml to make RAT happy. - Attila Doroszlai On Oct. 27, 2017, 3:19 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63360/ > --- > > (Updated Oct. 27, 2017, 3:19 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22315 > https://issues.apache.org/jira/browse/AMBARI-22315 > > > Repository: ambari > > > Description > --- > > its the next part of https://reviews.apache.org/r/63328/ > > contains some renames as well > > > Diffs > - > > ambari-logsearch/docker/docker-compose-zookeeper.yml a41eaf3 > ambari-logsearch/docker/solr.yml PRE-CREATION > ambari-logsearch/docker/solr662.yml PRE-CREATION > ambari-logsearch/docker/zookeeper.yml PRE-CREATION > > > Diff: https://reviews.apache.org/r/63360/diff/1/ > > > Testing > --- > > done with docker-compose up -d > > > Thanks, > > Oliver Szabo > >
Review Request 63361: AMBARI-22326. String formatting problems
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63361/ --- Review request for Ambari, Jonathan Hurley and Sebastian Toader. Bugs: AMBARI-22326 https://issues.apache.org/jira/browse/AMBARI-22326 Repository: ambari Description --- Trivial fix. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java e65693b47bf21a6e96cb40cdc0aa956b17477374 ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java fc657c1533d0e3f44b0974d9681867e14ceb6074 ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java cb44adba3adc68abd6287fce3f2edd4270405689 Diff: https://reviews.apache.org/r/63361/diff/1/ Testing --- ``` $ mvn -am -pl ambari-server clean compile ... [INFO] BUILD SUCCESS ``` Thanks, Attila Doroszlai
Review Request 63357: AMBARI-22322. Alert definition support for multi-service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63357/ --- Review request for Ambari, Jayush Luniya, Madhuvanthi Radhakrishnan, and Swapan Shridhar. Bugs: AMBARI-22322 https://issues.apache.org/jira/browse/AMBARI-22322 Repository: ambari Description --- * Introduce events for service type addition/removal to signal "first/last service of the same type" events. * Bind alert definition lifecycle to service type events instead of service events. Diffs - ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 07b70b6a507ac98ce6c89676ca40cd4f853970fe ambari-server/src/main/java/org/apache/ambari/server/events/ServiceGroupInstalledEvent.java e66d8a002a32336c49fc519a1ffe0676cf75ac12 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceGroupRemovedEvent.java b29a9a4fdc80d2d0bf0f61b10bed19d5483a34c0 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceInstalledEvent.java aa0a5ea8c30594208c574049cd98cd967db25a99 ambari-server/src/main/java/org/apache/ambari/server/events/ServiceTypeAddedEvent.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/events/ServiceTypeEvent.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/events/ServiceTypeRemovedEvent.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertServiceStateListener.java 41ae4e8e9ceab1e1f009b5c44b995b9357f8fe6c ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 5c6a82ffcfa53dfc1dee6fd97e9098c4f1ce56d2 ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 8fd878e6667c86c5989a213fb888989126fb77e2 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 025015a3e2bf30d5dfb714a46f3c1998a7183c6a ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 1e4d81f48b93abf137a6320d2ad31b6f33a6c392 ambari-server/src/main/java/org/apache/ambari/server/utils/EventBusSynchronizer.java c72ffbe032445fec725db04836d373c5ea15c603 ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 6ebd444e5541d84043ea8e51d1a2e50eb458b0ee ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java bd8be3b67225faa2bb33bedbd9fa42cb27dee694 Diff: https://reviews.apache.org/r/63357/diff/1/ Testing --- * Tested multiple service creation/removal manually * Adjusted `EventsTest` to cover the multiple services scenario Thanks, Attila Doroszlai
Re: Review Request 63240: RU tasks fail
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63240/#review189047 --- Ship it! Ship It! - Attila Doroszlai On Oct. 24, 2017, 9:58 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63240/ > --- > > (Updated Oct. 24, 2017, 9:58 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22295 > https://issues.apache.org/jira/browse/AMBARI-22295 > > > Repository: ambari > > > Description > --- > > File > "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py", > line 158, in > ExecuteUpgradeTasks().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 367, in execute > method(env) > File > "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py", > line 128, in actionexecute > base_dir = file_cache.get_service_base_dir(command_paths, > server_url_prefix) > File "/usr/lib/python2.6/site-packages/ambari_agent/FileCache.py", line > 75, in get_service_base_dir > service_subpath = > command['serviceLevelParams']['service_package_folder'] > KeyError: 'serviceLevelParams' > > > Diffs > - > > ambari-server/src/main/resources/custom_actions/scripts/ru_execute_tasks.py > 23a0cfb7f5 > > > Diff: https://reviews.apache.org/r/63240/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63243: Unable to Enable NameNode HA on branch-3.0-perf
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63243/#review189046
---
Ship it!
Ship It!
- Attila Doroszlai
On Oct. 24, 2017, 11:19 a.m., Andrew Onischuk wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63243/
> ---
>
> (Updated Oct. 24, 2017, 11:19 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid
> Wagle.
>
>
> Bugs: AMBARI-22296
> https://issues.apache.org/jira/browse/AMBARI-22296
>
>
> Repository: ambari
>
>
> Description
> ---
>
> I tried to Rolling update a new 2.5 cluster.
> 1\. I found that it requires NameNode HA so i started the Enable NameNode HA
> Wizard
> 2\. Configure Components page - Reconfigure HDFS - HDFS Client Install
> failed on all nodes:
>
>
>
>
> stderr:
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py",
> line 73, in
> HdfsClient().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 351, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py",
> line 35, in install
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/params.py",
> line 25, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py",
> line 319, in
> elif public_hostname.lower() in nn_host.lower():
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found
> in configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter
> 'public_hostname' was not found in configurations dictionary!
> stdout:
> 2017-10-20 11:06:56,480 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=None -> 2.5
> 2017-10-20 11:06:56,480 - Using hadoop conf dir:
> /usr/hdp/current/hadoop-client/conf
> User Group mapping (user_group) is missing in the hostLevelParams
> 2017-10-20 11:06:56,481 - Group['hadoop'] {}
> 2017-10-20 11:06:56,482 - Group['users'] {}
> 2017-10-20 11:06:56,482 - User['zookeeper'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
> 2017-10-20 11:06:56,483 - User['ambari-qa'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': ['users'], 'uid': None}
> 2017-10-20 11:06:56,483 - User['hdfs'] {'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
> 2017-10-20 11:06:56,484 - File['/var/lib/ambari-agent/tmp/changeUid.sh']
> {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> 2017-10-20 11:06:56,485 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh
> ambari-qa
> /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa
> 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
> 2017-10-20 11:06:56,490 - Skipping
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa
> 0'] due to not_if
> 2017-10-20 11:06:56,491 - Group['hdfs'] {}
> 2017-10-20 11:06:56,491 - User['hdfs'] {'fetch_nonlocal_groups': True,
> 'groups': ['hadoop', 'hdfs']}
> 2017-10-20 11:06:56,492 - FS Type:
> 2017-10-20 11:06:56,492 - Directory['/etc/hadoop'] {'mode': 0755}
> 2017-10-20 11:06:56,508 -
> File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content':
> InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
> 2017-10-20 11:06:56,509 -
> Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner':
> 'hdfs', 'group': 'hadoop', 'mode': 01777}
> 2017-10-20 11:06:56,524 - Repository['HDP-2.5-repo-1'] {'append_to_file':
> False, 'base_url':
> 'http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.5.3.0',
> 'action': ['create'], 'components': ['HDP', 'main'], 're
Re: Review Request 63181: Knox upgrade failed when migrating from IOP to HDP
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63181/ --- (Updated Oct. 20, 2017, 4:51 p.m.) Review request for Ambari, Jonathan Hurley and Nate Cole. Bugs: AMBARI-22279 https://issues.apache.org/jira/browse/AMBARI-22279 Repository: ambari Description --- ``` Execution of 'cp -R -p -f /usr/hdp/4.2.0.0/knox/data/. /usr/hdp/current/knox-server/data' returned 1. cp: cannot stat '/usr/hdp/4.2.0.0/knox/data/.': No such file or directory ``` Problems: 1. `commandParams/source_stack` is no longer sent, need to take source stack name from `upgradeSummary/services/KNOX/sourceStackId` 2. `cluster-env/stack_root` is overwritten during the upgrade Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py 7f2101d50b0820c7ae959bf4d4c481e36800c950 ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml 7c4a8a12e342672e92afde097a3c2a4f9a94615a Diff: https://reviews.apache.org/r/63181/diff/1/ Testing --- Manually tested upgrade from Ambari 2.2 + IOP 4.2 -> Ambari 2.5.2 -> Ambari 2.6 + HDP 2.6 Thanks, Attila Doroszlai
Review Request 63181: Knox upgrade failed when migrating from IOP to HDP
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63181/ --- Review request for Ambari, Jonathan Hurley and Nate Cole. Repository: ambari Description --- ``` Execution of 'cp -R -p -f /usr/hdp/4.2.0.0/knox/data/. /usr/hdp/current/knox-server/data' returned 1. cp: cannot stat '/usr/hdp/4.2.0.0/knox/data/.': No such file or directory ``` Problems: 1. `commandParams/source_stack` is no longer sent, need to take source stack name from `upgradeSummary/services/KNOX/sourceStackId` 2. `cluster-env/stack_root` is overwritten during the upgrade Diffs - ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py 7f2101d50b0820c7ae959bf4d4c481e36800c950 ambari-server/src/main/resources/stacks/BigInsights/4.0/configuration/cluster-env.xml 7c4a8a12e342672e92afde097a3c2a4f9a94615a Diff: https://reviews.apache.org/r/63181/diff/1/ Testing --- Manually tested upgrade from Ambari 2.2 + IOP 4.2 -> Ambari 2.5.2 -> Ambari 2.6 + HDP 2.6 Thanks, Attila Doroszlai
Re: Review Request 63034: AMBARI-22244. Update service component API call to work for second service instance
e5298dc629262aca4472c58d7d7abd2e7a63720a ambari-server/src/main/java/org/apache/ambari/server/mpack/MpackManager.java 0fcc28565d26ee0e9459b8329a0b64fbc4c2bde1 ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java d6876d9881116711eafb95eaa0e1901a6b870664 ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java e89f3dce89be7cd7d944bcabe40cd5d970097727 ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java df6437eab7a78afe734c78475b4c924944436d6c ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 70344b0a2e8428b5d3586dd77eeca20112bf7a8e ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ebc82ee3d8b6802994cb529d814d6a59f49bde7c ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 15f7048ab0f1682dd840d5e917b22148d0355e00 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java fcc07b6e030058058ab79718497cb95af3094186 ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 13ceeb914bf336d207e95b9ab018471b0c2e882d ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java acf0639f9359393dc3f569ad1f43484bd45213d0 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java 9c47e5d541e60d0ac843e0bc3341fea3467c9d7a ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 6190a03b8bae92519752a171f9d466bfb38e7218 ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostSummary.java 7edabc6df0db3feb978f2d2ba105cabbc1c34570 Diff: https://reviews.apache.org/r/63034/diff/2/ Changes: https://reviews.apache.org/r/63034/diff/1-2/ Testing --- Manual testing: * create 2 services for same service type (POST) * create a component for each service (POST) * query service (GET) * query component (GET) Thanks, Attila Doroszlai
Re: Review Request 63034: AMBARI-22244. Update service component API call to work for second service instance
d6876d9881116711eafb95eaa0e1901a6b870664 ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 58b0300b0b6cf15144d4bcd541671d962e074096 ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java df6437eab7a78afe734c78475b4c924944436d6c ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 70344b0a2e8428b5d3586dd77eeca20112bf7a8e ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 8f9d8e1b0b430e4c11e6935d2b7cbe3bedadd874 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 15f7048ab0f1682dd840d5e917b22148d0355e00 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 3bde889b4b9fa10203b38f06f18dd6ae707e9809 ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 13ceeb914bf336d207e95b9ab018471b0c2e882d ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java acf0639f9359393dc3f569ad1f43484bd45213d0 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java 9c47e5d541e60d0ac843e0bc3341fea3467c9d7a ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 6190a03b8bae92519752a171f9d466bfb38e7218 ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostSummary.java 7edabc6df0db3feb978f2d2ba105cabbc1c34570 Diff: https://reviews.apache.org/r/63034/diff/1/ Testing (updated) --- Manual testing: * create 2 services for same service type (POST) * create a component for each service (POST) * query service (GET) * query component (GET) Thanks, Attila Doroszlai
Re: Review Request 62930: Non-root ambari user causes Express Upgrade to fail during checking HDFS safe mode phase
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62930/#review188291
---
Fix it, then Ship it!
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/package/scripts/namenode_upgrade.py
Line 174 (original), 174 (patched)
<https://reviews.apache.org/r/62930/#comment265291>
Please use `{safemode_check}` for the first part of the command instead of
repeating the contents.
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/package/scripts/namenode_upgrade.py
Line 175 (original), 175 (patched)
<https://reviews.apache.org/r/62930/#comment265289>
Need to pass `user=user` to the `call` function for the command to be run
as the HDFS user.
- Attila Doroszlai
On Oct. 12, 2017, 4:22 p.m., Piyush Chavhan wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62930/
> ---
>
> (Updated Oct. 12, 2017, 4:22 p.m.)
>
>
> Review request for Ambari.
>
>
> Bugs: AMBARI-22187
> https://issues.apache.org/jira/browse/AMBARI-22187
>
>
> Repository: ambari
>
>
> Description
> ---
>
> If ambari-server and ambari-agents running using non-root user and tried to
> upgrade from BigInsights stack 4.1 to 4.2.5. Perform Backups fails due to
> "get safemode state" fails for NameNode
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/package/scripts/namenode_upgrade.py
> 5969fcf
>
>
> Diff: https://reviews.apache.org/r/62930/diff/1/
>
>
> Testing
> ---
>
> Tested this change on the running cluster by performing express upgrade from
> BigInsights 4.1 stack to 4.2.5 with HDFS and Zookeeper services running on
> the cluster.
>
> UTs are still running for this.
>
>
> Thanks,
>
> Piyush Chavhan
>
>
Re: Review Request 63036: AMBARI-22234 Optimizing Ranger KMS imports
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63036/#review188163
---
Ship it!
Ship It!
- Attila Doroszlai
On Oct. 16, 2017, 6:54 p.m., Mugdha Varadkar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63036/
> ---
>
> (Updated Oct. 16, 2017, 6:54 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Gautam Borad, and Jonathan
> Hurley.
>
>
> Bugs: AMBARI-22234
> https://issues.apache.org/jira/browse/AMBARI-22234
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Updating Ranger Kms script file and using proper methods for importing
> modules.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py
> fcf2478
>
> ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py
> 4c313c4
>
>
> Diff: https://reviews.apache.org/r/63036/diff/1/
>
>
> Testing
> ---
>
> Running tests for stack:None service:RANGER_KMS
> 2017-10-13 17:44:18,232 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,255 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_db_flavor_0_5_0_2_3 (test_db_flavor_config.TestDbFlavorConfig) ...
> 2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> 2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> ok
> 2017-10-13 17:44:18,275 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,298 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_db_flavor_1_0_0_3_0 (test_db_flavor_config.TestDbFlavorConfig) ...
> 2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> 2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> ok
> 2017-10-13 17:44:18,314 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,337 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_unsupported_db_flavor_0_5_0_2_3
> (test_db_flavor_config.TestDbFlavorConfig) ... ok
> 2017-10-13 17:44:18,344 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,367 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_unsupported_db_flavor_1_0_0_3_0
> (test_db_flavor_config.TestDbFlavorConfig) ... ok
>
> --
> Ran 4 tests in 0.147s
>
> OK
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Review Request 63034: AMBARI-22244. Update service component API call to work for second service instance
d6876d9881116711eafb95eaa0e1901a6b870664 ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 58b0300b0b6cf15144d4bcd541671d962e074096 ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java df6437eab7a78afe734c78475b4c924944436d6c ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 70344b0a2e8428b5d3586dd77eeca20112bf7a8e ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 8f9d8e1b0b430e4c11e6935d2b7cbe3bedadd874 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 15f7048ab0f1682dd840d5e917b22148d0355e00 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 3bde889b4b9fa10203b38f06f18dd6ae707e9809 ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 13ceeb914bf336d207e95b9ab018471b0c2e882d ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java acf0639f9359393dc3f569ad1f43484bd45213d0 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java 9c47e5d541e60d0ac843e0bc3341fea3467c9d7a ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 6190a03b8bae92519752a171f9d466bfb38e7218 ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostSummary.java 7edabc6df0db3feb978f2d2ba105cabbc1c34570 Diff: https://reviews.apache.org/r/63034/diff/1/ Testing --- Manual testing: * create 2 services for same service type (POST) * create a component for each service (POST) * query service (GET) * query component (GET) More API testing in progress. Thanks, Attila Doroszlai
Re: Review Request 62998: In case only Hive Server Interactive is deployed with no Hive Server in the cluster the hive metastore warehouse directory is not created on DFS.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62998/#review188073 --- Ship it! Ship It! - Attila Doroszlai On Oct. 14, 2017, 8:05 a.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62998/ > --- > > (Updated Oct. 14, 2017, 8:05 a.m.) > > > Review request for Ambari, Attila Doroszlai, Laszlo Puskas, Robert Levas, and > Swapan Shridhar. > > > Bugs: AMBARI-22243 > https://issues.apache.org/jira/browse/AMBARI-22243 > > > Repository: ambari > > > Description > --- > > The ware house directory creation logic was part only of HiveServer scripts. > The same logic had to be added to HiveServer Interactive scripts in order to > ensure that the warehouse directory is created if solely HiveServer > Interactive is deployed to the cluster. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_interactive.py > 2ed3e3a > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_interactive.py > 2ed3e3a > ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py > 3dc78ab > > > Diff: https://reviews.apache.org/r/62998/diff/1/ > > > Testing > --- > > Manual testing using Blueprints (currently only with Blueprints it's possible > to deploy HiveServer Interactive with no HiveServer in the cluster) > > > Unit tests: > *mvn test -DskipSurefireTests -am -pl ambari-server* > > ``` > Total run:1204 > Total errors:0 > Total failures:0 > OK > [INFO] > [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server --- > [INFO] Starting audit... > Audit done. > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main SUCCESS [ 0.473 > s] > [INFO] Apache Ambari Project POM .. SUCCESS [ 0.003 > s] > [INFO] Ambari Views ... SUCCESS [ 1.727 > s] > [INFO] ambari-utility . SUCCESS [ 2.642 > s] > [INFO] ambari-metrics . SUCCESS [ 0.185 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 0.351 > s] > [INFO] Ambari Service Advisor . SUCCESS [ 0.374 > s] > [INFO] Ambari Server .. SUCCESS [01:32 > min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > ``` > > > Thanks, > > Sebastian Toader > >
Re: Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
> On Oct. 9, 2017, 4:23 p.m., Sebastian Toader wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Line 1374 (original), 1420 (patched) > > <https://reviews.apache.org/r/62809/diff/2/?file=1849509#file1849509line1420> > > > > Can you do the same for `MultipleHostTopologyUpdater` and other > > updaters which currently process hostgroup plcaeholders? `MultipleHostTopologyUpdater` has much more complex logic for hostgroups, which I'd rather not change now. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62809/#review187412 ------- On Oct. 9, 2017, 2:43 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62809/ > --- > > (Updated Oct. 9, 2017, 2:43 p.m.) > > > Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, Sandor Magyari, > Swapan Shridhar, and Sebastian Toader. > > > Bugs: AMBARI-22159 > https://issues.apache.org/jira/browse/AMBARI-22159 > > > Repository: ambari > > > Description > --- > > Replace placeholders in `metastore_hostname`, > `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host` > when creating cluster via blueprint. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 68d6349db0207ab6df44b4b0a341c77ecabc46ea > > > Diff: https://reviews.apache.org/r/62809/diff/2/ > > > Testing > --- > > Tested manually on local cluster. > > Unit tests pass: `Tests run: 4900, Failures: 0, Errors: 0, Skipped: 36` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62809/ --- (Updated Oct. 9, 2017, 2:43 p.m.) Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, Sandor Magyari, Swapan Shridhar, and Sebastian Toader. Changes --- extracted property updater that only replaces `%HOSTGROUP::name%` placeholders Bugs: AMBARI-22159 https://issues.apache.org/jira/browse/AMBARI-22159 Repository: ambari Description --- Replace placeholders in `metastore_hostname`, `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host` when creating cluster via blueprint. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d6349db0207ab6df44b4b0a341c77ecabc46ea Diff: https://reviews.apache.org/r/62809/diff/2/ Changes: https://reviews.apache.org/r/62809/diff/1-2/ Testing --- Tested manually on local cluster. Unit tests pass: `Tests run: 4900, Failures: 0, Errors: 0, Skipped: 36` Thanks, Attila Doroszlai
Re: Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
> On Oct. 9, 2017, 11:41 a.m., Sebastian Toader wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Lines 2782 (patched) > > <https://reviews.apache.org/r/62809/diff/1/?file=1847507#file1847507line2782> > > > > Maybe we should introduce a property updater that handles only > > `%HOSTGROUP::name%` constructs. Good idea. > On Oct. 9, 2017, 11:41 a.m., Sebastian Toader wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Lines 2784 (patched) > > <https://reviews.apache.org/r/62809/diff/1/?file=1847507#file1847507line2784> > > > > Why property updater for `druid.zk.service.host` is not needed any more? I don't understand the question. The updater is being added by the patch, not removed. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62809/#review187393 ------- On Oct. 9, 2017, 9:16 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62809/ > --- > > (Updated Oct. 9, 2017, 9:16 a.m.) > > > Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, Sandor Magyari, > Swapan Shridhar, and Sebastian Toader. > > > Bugs: AMBARI-22159 > https://issues.apache.org/jira/browse/AMBARI-22159 > > > Repository: ambari > > > Description > --- > > Replace placeholders in `metastore_hostname`, > `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host` > when creating cluster via blueprint. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 68d6349db0207ab6df44b4b0a341c77ecabc46ea > > > Diff: https://reviews.apache.org/r/62809/diff/1/ > > > Testing > --- > > Tested manually on local cluster. > > Unit tests pass: `Tests run: 4900, Failures: 0, Errors: 0, Skipped: 36` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
> On Oct. 7, 2017, 2 p.m., Oliver Szabo wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Lines 2783 (patched) > > <https://reviews.apache.org/r/62809/diff/1/?file=1847507#file1847507line2783> > > > > can these variables calculated at params.py in the stack code instead > > of putting to the BlueprintConfigurationProcessor? (using jinja2 variables > > at configurations - you can get the proper hosts from command.json input). > > just because the ambari server java code could be independent from druid. > > im not too big fan of this class, we should get rid of it sooner or later > > > > but you can drop this issue if zou do not want to do it, it's just a > > reccomendation from me. I agree that this class stinks. However, currently no `%HOSTGROUP::...%` substitution occurs in Python code. I guess it shouldn't be introduced at this point before 2.6 release. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62809/#review187335 ------- On Oct. 6, 2017, 5:35 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62809/ > --- > > (Updated Oct. 6, 2017, 5:35 p.m.) > > > Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, and Swapan > Shridhar. > > > Bugs: AMBARI-22159 > https://issues.apache.org/jira/browse/AMBARI-22159 > > > Repository: ambari > > > Description > --- > > Replace placeholders in `metastore_hostname`, > `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host` > when creating cluster via blueprint. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 68d6349db0207ab6df44b4b0a341c77ecabc46ea > > > Diff: https://reviews.apache.org/r/62809/diff/1/ > > > Testing > --- > > Tested manually on local cluster. > > Unit tests pass: `Tests run: 4900, Failures: 0, Errors: 0, Skipped: 36` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 62805: AMBARI-22155. Intermittent failure of FlumeTimelineMetricsSinkTest
> On Oct. 6, 2017, 6:45 p.m., Sid Wagle wrote: > > ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java > > Lines 84 (patched) > > <https://reviews.apache.org/r/62805/diff/1/?file=1847470#file1847470line97> > > > > Missing expectLastCall() ? Calling a method on the mock sets up the expectation, so `expectLastCall()` is only needed if you want to tweak the expectation (eg. return value, call count, or exception thrown). However, it turned out that `PowerMock.verifyAll()` did not verify expectations of `executor`, because it is not "maintained" by PowerMock. This allowed the test to pass even with the `stop()` call removed. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62805/#review187278 ------- On Oct. 6, 2017, 10:39 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62805/ > --- > > (Updated Oct. 6, 2017, 10:39 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle. > > > Bugs: AMBARI-22155 > https://issues.apache.org/jira/browse/AMBARI-22155 > > > Repository: ambari > > > Description > --- > > Mock the executor to eliminate nondeterminism caused by threads. > > > Diffs > - > > > ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java > 62779070419894064cb39ea5cc5da012572ff4cb > > ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java > bd4ae6af88ee13f4ba6a3f14ac8c138b72d63c9d > > > Diff: https://reviews.apache.org/r/62805/diff/2/ > > > Testing > --- > > ``` > $ mvn -am -pl ambari-metrics/ambari-metrics-flume-sink clean test > ... > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 > ... > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main SUCCESS [ 6.159 > s] > [INFO] Apache Ambari Project POM .. SUCCESS [ 0.036 > s] > [INFO] ambari-utility . SUCCESS [ 3.880 > s] > [INFO] ambari-metrics . SUCCESS [ 0.594 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 4.240 > s] > [INFO] Ambari Metrics Flume Sink .. SUCCESS [ 4.697 > s] > [INFO] > > [INFO] BUILD SUCCESS > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 62805: AMBARI-22155. Intermittent failure of FlumeTimelineMetricsSinkTest
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62805/ --- (Updated Oct. 6, 2017, 10:39 p.m.) Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle. Changes --- make sure executor mock is verified Bugs: AMBARI-22155 https://issues.apache.org/jira/browse/AMBARI-22155 Repository: ambari Description --- Mock the executor to eliminate nondeterminism caused by threads. Diffs (updated) - ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java 62779070419894064cb39ea5cc5da012572ff4cb ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java bd4ae6af88ee13f4ba6a3f14ac8c138b72d63c9d Diff: https://reviews.apache.org/r/62805/diff/2/ Changes: https://reviews.apache.org/r/62805/diff/1-2/ Testing --- ``` $ mvn -am -pl ambari-metrics/ambari-metrics-flume-sink clean test ... Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 ... [INFO] Reactor Summary: [INFO] [INFO] Ambari Main SUCCESS [ 6.159 s] [INFO] Apache Ambari Project POM .. SUCCESS [ 0.036 s] [INFO] ambari-utility . SUCCESS [ 3.880 s] [INFO] ambari-metrics . SUCCESS [ 0.594 s] [INFO] Ambari Metrics Common .. SUCCESS [ 4.240 s] [INFO] Ambari Metrics Flume Sink .. SUCCESS [ 4.697 s] [INFO] [INFO] BUILD SUCCESS ``` Thanks, Attila Doroszlai
Re: Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
> On Oct. 6, 2017, 9:47 p.m., Nishant Bangarwa wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
> > Lines 2782 (patched)
> > <https://reviews.apache.org/r/62809/diff/1/?file=1847507#file1847507line2782>
> >
> > this seems odd, metadata store is not always installed on the same host
> > as coordinator.
> > Fwiw, metadata store for druid is not installed/managed by ambari, user
> > needs to set it up and provide valid hostname and uri.
For explicit `%HOSTGROUP::some_host_group%` the hostname from the given host
group is substituted, component is ignored.
Component ("DRUID_COORDINATOR") is relevant only if property value contains
"localhost".
- Attila
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62809/#review187297
-------
On Oct. 6, 2017, 5:35 p.m., Attila Doroszlai wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62809/
> ---
>
> (Updated Oct. 6, 2017, 5:35 p.m.)
>
>
> Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, and Swapan
> Shridhar.
>
>
> Bugs: AMBARI-22159
> https://issues.apache.org/jira/browse/AMBARI-22159
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Replace placeholders in `metastore_hostname`,
> `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host`
> when creating cluster via blueprint.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
> 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 68d6349db0207ab6df44b4b0a341c77ecabc46ea
>
>
> Diff: https://reviews.apache.org/r/62809/diff/1/
>
>
> Testing
> ---
>
> Tested manually on local cluster.
>
> Unit tests pass: `Tests run: 4900, Failures: 0, Errors: 0, Skipped: 36`
>
>
> Thanks,
>
> Attila Doroszlai
>
>
Re: Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62809/ --- (Updated Oct. 6, 2017, 5:35 p.m.) Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, and Swapan Shridhar. Bugs: AMBARI-22159 https://issues.apache.org/jira/browse/AMBARI-22159 Repository: ambari Description --- Replace placeholders in `metastore_hostname`, `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host` when creating cluster via blueprint. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d6349db0207ab6df44b4b0a341c77ecabc46ea Diff: https://reviews.apache.org/r/62809/diff/1/ Testing (updated) --- Tested manually on local cluster. Unit tests pass: `Tests run: 4900, Failures: 0, Errors: 0, Skipped: 36` Thanks, Attila Doroszlai
Review Request 62809: AMBARI-22159. Replace hostgroup vars for Druid
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62809/ --- Review request for Ambari, Nishant Bangarwa, Slim Bouguerra, and Swapan Shridhar. Bugs: AMBARI-22159 https://issues.apache.org/jira/browse/AMBARI-22159 Repository: ambari Description --- Replace placeholders in `metastore_hostname`, `druid.metadata.storage.connector.connectURI` and `druid.zk.service.host` when creating cluster via blueprint. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 5a6e2cc0de7dc0c0f4499624e21c43bbc8d24b54 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d6349db0207ab6df44b4b0a341c77ecabc46ea Diff: https://reviews.apache.org/r/62809/diff/1/ Testing --- Tested manually on local cluster. Unit tests: `BlueprintConfigurationProcessorTest` passed, others pending. Thanks, Attila Doroszlai
Review Request 62790: AMBARI-22141. Intermittent failure of test_stack_advisor_perf
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62790/ --- Review request for Ambari, Laszlo Puskas, Sandor Magyari, and Sebastian Toader. Bugs: AMBARI-22141 https://issues.apache.org/jira/browse/AMBARI-22141 Repository: ambari Description --- * Increased time allowed to 0.2s * Load more recent stack advisor implementations, too * Cleaned up unused code and duplication Diffs - ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 82cef1bfd018a26ef007000538b7e1eb2aa028ed Diff: https://reviews.apache.org/r/62790/diff/1/ Testing --- ``` $ mvn -am -pl ambari-server -DskipSurefireTests clean test ... Total run:1203 Total errors:0 Total failures:0 OK ... [INFO] BUILD SUCCESS ``` Thanks, Attila Doroszlai
Re: Review Request 62764: AMBARI-22127. Installation of stack selector fails on Debian
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62764/ --- (Updated Oct. 5, 2017, 4:17 p.m.) Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Sebastian Toader. Changes --- also tested on SuSE11 Bugs: AMBARI-22127 https://issues.apache.org/jira/browse/AMBARI-22127 Repository: ambari Description --- Need to pass both repo IDs and repo file names via `use_repos`, because Debian/Ubuntu selects specific repos by file name, while RedHat/SuSE by repo ID. Diffs - ambari-agent/src/test/python/resource_management/TestPackageResource.py 49e3adf6782021977ff3edce850da35e3adc7116 ambari-common/src/main/python/resource_management/core/providers/package/__init__.py 94047571b5113716ee041e890fea559b4ce263ee ambari-common/src/main/python/resource_management/core/providers/package/apt.py f6a5538a30cf86eb4125ff088a1a8ca24c5adece ambari-common/src/main/python/resource_management/core/providers/package/choco.py db552964279f509de90d2fde884bfc0fea12f7b0 ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py a4a55da7c9c5e9ac554d91edd19e9cb668615671 ambari-common/src/main/python/resource_management/core/providers/package/zypper.py 5b8e5ab94091bf181e97defd6a13dea4e3a46863 ambari-common/src/main/python/resource_management/core/resources/packaging.py e3adc30f46abd3f33670f817421d88270b3e7ade ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py f1074ee7b3748cfedb3e1a813be52df894cff97f ambari-server/src/main/resources/custom_actions/scripts/install_packages.py dcf354447756d2e1c826feb8515cfe3b9711f19d ambari-server/src/test/python/custom_actions/TestInstallPackages.py 0f303bee541ea3ac06fadb67eb44f5aab97fa1d5 Diff: https://reviews.apache.org/r/62764/diff/1/ Testing (updated) --- Tested package installation (both pre-upgrade "Install Version" and regular "Add Service") on Debian 7.6, CentOS 6.4 and SuSE11. Unit tests pass (both branch-2.6 and trunk): ``` $ mvn -am -pl ambari-agent,ambari-server -DskipSurefireTests test ... Total run:1192 Total errors:0 Total failures:0 OK ... Ran 453 tests in 137.865s OK ... [INFO] Ambari Server .. SUCCESS [01:28 min] [INFO] Ambari Agent ... SUCCESS [02:21 min] ``` Thanks, Attila Doroszlai
Re: Review Request 62764: AMBARI-22127. Installation of stack selector fails on Debian
> On Oct. 4, 2017, 5:18 p.m., Nate Cole wrote: > > ambari-common/src/main/python/resource_management/core/providers/package/__init__.py > > Lines 42-43 (original), 42-43 (patched) > > <https://reviews.apache.org/r/62764/diff/1/?file=1845759#file1845759line42> > > > > This type of change from a set to a dictionary is very risky at this > > point for 2.6. Is there anyway to just fix the apt code for this? > > > > In addition, make sure you run this through CI in order to make sure > > acceptance tests won't go crazy. I doubt this breaks anything. It seems `use_repos` has been "dead code" for at least 2 years: it was broken on Debian/Ubuntu since May 2015 and nobody noticed, until AMBARI-22076 introduced code that started using it. We could probably check the OS in the OS-agnostic code and pass different sets in each case. Sounds like an ugly hack to me. > On Oct. 4, 2017, 5:18 p.m., Nate Cole wrote: > > ambari-common/src/main/python/resource_management/core/providers/package/apt.py > > Lines 351 (patched) > > <https://reviews.apache.org/r/62764/diff/1/?file=1845760#file1845760line351> > > > > This % syntax is old, we typically use string.format() This line is old code, just moved around a bit. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62764/#review187101 --- On Oct. 4, 2017, 4:32 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62764/ > --- > > (Updated Oct. 4, 2017, 4:32 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan > Hurley, Nate Cole, and Sebastian Toader. > > > Bugs: AMBARI-22127 > https://issues.apache.org/jira/browse/AMBARI-22127 > > > Repository: ambari > > > Description > --- > > Need to pass both repo IDs and repo file names via `use_repos`, because > Debian/Ubuntu selects specific repos by file name, while RedHat/SuSE by repo > ID. > > > Diffs > - > > ambari-agent/src/test/python/resource_management/TestPackageResource.py > 49e3adf6782021977ff3edce850da35e3adc7116 > > ambari-common/src/main/python/resource_management/core/providers/package/__init__.py > 94047571b5113716ee041e890fea559b4ce263ee > > ambari-common/src/main/python/resource_management/core/providers/package/apt.py > f6a5538a30cf86eb4125ff088a1a8ca24c5adece > > ambari-common/src/main/python/resource_management/core/providers/package/choco.py > db552964279f509de90d2fde884bfc0fea12f7b0 > > ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py > a4a55da7c9c5e9ac554d91edd19e9cb668615671 > > ambari-common/src/main/python/resource_management/core/providers/package/zypper.py > 5b8e5ab94091bf181e97defd6a13dea4e3a46863 > > ambari-common/src/main/python/resource_management/core/resources/packaging.py > e3adc30f46abd3f33670f817421d88270b3e7ade > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1074ee7b3748cfedb3e1a813be52df894cff97f > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > dcf354447756d2e1c826feb8515cfe3b9711f19d > ambari-server/src/test/python/custom_actions/TestInstallPackages.py > 0f303bee541ea3ac06fadb67eb44f5aab97fa1d5 > > > Diff: https://reviews.apache.org/r/62764/diff/1/ > > > Testing > --- > > Tested package installation (both pre-upgrade "Install Version" and regular > "Add Service") on Debian 7.6 and CentOS 6.4. > > Unit tests pass (both branch-2.6 and trunk): > > ``` > $ mvn -am -pl ambari-agent,ambari-server -DskipSurefireTests test > ... > Total run:1192 > Total errors:0 > Total failures:0 > OK > ... > Ran 453 tests in 137.865s > > OK > ... > [INFO] Ambari Server .. SUCCESS [01:28 > min] > [INFO] Ambari Agent ... SUCCESS [02:21 > min] > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 62764: AMBARI-22127. Installation of stack selector fails on Debian
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62764/ --- (Updated Oct. 4, 2017, 4:32 p.m.) Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Sebastian Toader. Bugs: AMBARI-22127 https://issues.apache.org/jira/browse/AMBARI-22127 Repository: ambari Description --- Need to pass both repo IDs and repo file names via `use_repos`, because Debian/Ubuntu selects specific repos by file name, while RedHat/SuSE by repo ID. Diffs - ambari-agent/src/test/python/resource_management/TestPackageResource.py 49e3adf6782021977ff3edce850da35e3adc7116 ambari-common/src/main/python/resource_management/core/providers/package/__init__.py 94047571b5113716ee041e890fea559b4ce263ee ambari-common/src/main/python/resource_management/core/providers/package/apt.py f6a5538a30cf86eb4125ff088a1a8ca24c5adece ambari-common/src/main/python/resource_management/core/providers/package/choco.py db552964279f509de90d2fde884bfc0fea12f7b0 ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py a4a55da7c9c5e9ac554d91edd19e9cb668615671 ambari-common/src/main/python/resource_management/core/providers/package/zypper.py 5b8e5ab94091bf181e97defd6a13dea4e3a46863 ambari-common/src/main/python/resource_management/core/resources/packaging.py e3adc30f46abd3f33670f817421d88270b3e7ade ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py f1074ee7b3748cfedb3e1a813be52df894cff97f ambari-server/src/main/resources/custom_actions/scripts/install_packages.py dcf354447756d2e1c826feb8515cfe3b9711f19d ambari-server/src/test/python/custom_actions/TestInstallPackages.py 0f303bee541ea3ac06fadb67eb44f5aab97fa1d5 Diff: https://reviews.apache.org/r/62764/diff/1/ Testing (updated) --- Tested package installation (both pre-upgrade "Install Version" and regular "Add Service") on Debian 7.6 and CentOS 6.4. Unit tests pass (both branch-2.6 and trunk): ``` $ mvn -am -pl ambari-agent,ambari-server -DskipSurefireTests test ... Total run:1192 Total errors:0 Total failures:0 OK ... Ran 453 tests in 137.865s OK ... [INFO] Ambari Server .. SUCCESS [01:28 min] [INFO] Ambari Agent ... SUCCESS [02:21 min] ``` Thanks, Attila Doroszlai
Review Request 62764: AMBARI-22127. Installation of stack selector fails on Debian
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62764/ --- Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Sebastian Toader. Bugs: AMBARI-22127 https://issues.apache.org/jira/browse/AMBARI-22127 Repository: ambari Description --- Need to pass both repo IDs and repo file names via `use_repos`, because Debian/Ubuntu selects specific repos by file name, while RedHat/SuSE by repo ID. Diffs - ambari-agent/src/test/python/resource_management/TestPackageResource.py 49e3adf6782021977ff3edce850da35e3adc7116 ambari-common/src/main/python/resource_management/core/providers/package/__init__.py 94047571b5113716ee041e890fea559b4ce263ee ambari-common/src/main/python/resource_management/core/providers/package/apt.py f6a5538a30cf86eb4125ff088a1a8ca24c5adece ambari-common/src/main/python/resource_management/core/providers/package/choco.py db552964279f509de90d2fde884bfc0fea12f7b0 ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py a4a55da7c9c5e9ac554d91edd19e9cb668615671 ambari-common/src/main/python/resource_management/core/providers/package/zypper.py 5b8e5ab94091bf181e97defd6a13dea4e3a46863 ambari-common/src/main/python/resource_management/core/resources/packaging.py e3adc30f46abd3f33670f817421d88270b3e7ade ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py f1074ee7b3748cfedb3e1a813be52df894cff97f ambari-server/src/main/resources/custom_actions/scripts/install_packages.py dcf354447756d2e1c826feb8515cfe3b9711f19d ambari-server/src/test/python/custom_actions/TestInstallPackages.py 0f303bee541ea3ac06fadb67eb44f5aab97fa1d5 Diff: https://reviews.apache.org/r/62764/diff/1/ Testing --- Tested package installation (both pre-upgrade "Install Version" and regular "Add Service") on Debian 7.6 and CentOS 6.4. Unit tests (on `branch-2.6`): ``` $ mvn -am -pl ambari-agent,ambari-server -DskipSurefireTests test ... Total run:1192 Total errors:0 Total failures:0 OK ... Ran 453 tests in 137.865s OK ... [INFO] Ambari Server .. SUCCESS [01:28 min] [INFO] Ambari Agent ... SUCCESS [02:21 min] ``` Unit tests of Ambari Agent on `trunk` are wrong due to an unrelated change. Thanks, Attila Doroszlai
Re: Review Request 62750: Cache becomes corrupt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62750/#review186949 --- Ship it! Ship It! - Attila Doroszlai On Oct. 3, 2017, 3:35 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62750/ > --- > > (Updated Oct. 3, 2017, 3:35 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22119 > https://issues.apache.org/jira/browse/AMBARI-22119 > > > Repository: ambari > > > Description > --- > > Agent topology file cache became corrupt after writing of topology to it. It > contained some gibberish. I think it might be a rare simplejson bug. The file > was not trimmed correctly, during rewritting. > > This happened only once on perf cluster. But worth a preventive fix. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/ClusterCache.py 40b32886cf > > > Diff: https://reviews.apache.org/r/62750/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 62723: AMBARI-22113. Unit test fails with Python 2.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62723/ --- (Updated Oct. 2, 2017, 8:12 p.m.) Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sebastian Toader, and Vitalyi Brodetskyi. Summary (updated) - AMBARI-22113. Unit test fails with Python 2.6 Bugs: AMBARI-22113 https://issues.apache.org/jira/browse/AMBARI-22113 Repository: ambari Description --- Eliminate/tweak Python 2.7+ constructs: * multiple contexts in `with` * `assertIsNone` * `assertRaises` * `unittest.util` Diffs - ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 86821bf47a757284f02ed5049fb29ccb71451340 ambari-server/src/test/python/TestUpgradeSummary.py 7606867f87f580e081c466ec889a6baa122b6c7a ambari-server/src/test/python/common-services/HIVE/test_jdbc_driver_config.py e4d81b1bf650e15ac730c0e5d889ba2f98c32308 ambari-server/src/test/python/common-services/RANGER/test_db_flavor_config.py 568e3fd7bb99ba888755382bce89065ea7d40914 ambari-server/src/test/python/common-services/RANGER_KMS/test_db_flavor_config.py 48654ee18ad80aed35c3cab3d1ccbe6783be6b43 ambari-server/src/test/python/common-services/SQOOP/test_jdbc_driver_config.py 7bb809a0a4e04565a1b201f08db8e0a93c3fd42c ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 0cc9ad0a603f5988e7f3af76babc9ad677c58149 ambari-server/src/test/python/stacks/utils/RMFTestCase.py 291f22b0fb1907cda4df4276aeff8a9df6d32b30 Diff: https://reviews.apache.org/r/62723/diff/1/ Testing --- Tested with Python 2.6: ``` $ mvn -am -pl ambari-server,ambari-agent -Drat.skip -Dcheckstyle.skip -DskipSurefireTests clean test ... Total run:1202 Total errors:0 Total failures:0 ... Ran 454 tests in 22.430s ... [INFO] Ambari Server .. FAILURE [01:41 min] [INFO] Ambari Agent ... SUCCESS [ 28.188 s] ... ``` and Python 2.7: ``` $ mvn -am -pl ambari-server,ambari-agent -Drat.skip -Dcheckstyle.skip -DskipSurefireTests clean test ... Total run:1202 Total errors:0 Total failures:0 ... Ran 457 tests in 16.645s ... [INFO] Ambari Server .. SUCCESS [01:35 min] [INFO] Ambari Agent ... SUCCESS [ 20.611 s] ``` Thanks, Attila Doroszlai
Re: Review Request 62723: Unit test fails with Python 2.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62723/ --- (Updated Oct. 2, 2017, 2:04 p.m.) Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sebastian Toader, and Vitalyi Brodetskyi. Summary (updated) - Unit test fails with Python 2.6 Repository: ambari Description --- Eliminate/tweak Python 2.7+ constructs: * multiple contexts in `with` * `assertIsNone` * `assertRaises` * `unittest.util` Diffs - ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 86821bf47a757284f02ed5049fb29ccb71451340 ambari-server/src/test/python/TestUpgradeSummary.py 7606867f87f580e081c466ec889a6baa122b6c7a ambari-server/src/test/python/common-services/HIVE/test_jdbc_driver_config.py e4d81b1bf650e15ac730c0e5d889ba2f98c32308 ambari-server/src/test/python/common-services/RANGER/test_db_flavor_config.py 568e3fd7bb99ba888755382bce89065ea7d40914 ambari-server/src/test/python/common-services/RANGER_KMS/test_db_flavor_config.py 48654ee18ad80aed35c3cab3d1ccbe6783be6b43 ambari-server/src/test/python/common-services/SQOOP/test_jdbc_driver_config.py 7bb809a0a4e04565a1b201f08db8e0a93c3fd42c ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 0cc9ad0a603f5988e7f3af76babc9ad677c58149 ambari-server/src/test/python/stacks/utils/RMFTestCase.py 291f22b0fb1907cda4df4276aeff8a9df6d32b30 Diff: https://reviews.apache.org/r/62723/diff/1/ Testing --- Tested with Python 2.6: ``` $ mvn -am -pl ambari-server,ambari-agent -Drat.skip -Dcheckstyle.skip -DskipSurefireTests clean test ... Total run:1202 Total errors:0 Total failures:0 ... Ran 454 tests in 22.430s ... [INFO] Ambari Server .. FAILURE [01:41 min] [INFO] Ambari Agent ... SUCCESS [ 28.188 s] ... ``` and Python 2.7: ``` $ mvn -am -pl ambari-server,ambari-agent -Drat.skip -Dcheckstyle.skip -DskipSurefireTests clean test ... Total run:1202 Total errors:0 Total failures:0 ... Ran 457 tests in 16.645s ... [INFO] Ambari Server .. SUCCESS [01:35 min] [INFO] Ambari Agent ... SUCCESS [ 20.611 s] ``` Thanks, Attila Doroszlai
Review Request 62723: BUG-89200. Unit test fails with Python 2.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62723/ --- Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sebastian Toader, and Vitalyi Brodetskyi. Repository: ambari Description --- Eliminate/tweak Python 2.7+ constructs: * multiple contexts in `with` * `assertIsNone` * `assertRaises` * `unittest.util` Diffs - ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 86821bf47a757284f02ed5049fb29ccb71451340 ambari-server/src/test/python/TestUpgradeSummary.py 7606867f87f580e081c466ec889a6baa122b6c7a ambari-server/src/test/python/common-services/HIVE/test_jdbc_driver_config.py e4d81b1bf650e15ac730c0e5d889ba2f98c32308 ambari-server/src/test/python/common-services/RANGER/test_db_flavor_config.py 568e3fd7bb99ba888755382bce89065ea7d40914 ambari-server/src/test/python/common-services/RANGER_KMS/test_db_flavor_config.py 48654ee18ad80aed35c3cab3d1ccbe6783be6b43 ambari-server/src/test/python/common-services/SQOOP/test_jdbc_driver_config.py 7bb809a0a4e04565a1b201f08db8e0a93c3fd42c ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 0cc9ad0a603f5988e7f3af76babc9ad677c58149 ambari-server/src/test/python/stacks/utils/RMFTestCase.py 291f22b0fb1907cda4df4276aeff8a9df6d32b30 Diff: https://reviews.apache.org/r/62723/diff/1/ Testing --- Tested with Python 2.6: ``` $ mvn -am -pl ambari-server,ambari-agent -Drat.skip -Dcheckstyle.skip -DskipSurefireTests clean test ... Total run:1202 Total errors:0 Total failures:0 ... Ran 454 tests in 22.430s ... [INFO] Ambari Server .. FAILURE [01:41 min] [INFO] Ambari Agent ... SUCCESS [ 28.188 s] ... ``` and Python 2.7: ``` $ mvn -am -pl ambari-server,ambari-agent -Drat.skip -Dcheckstyle.skip -DskipSurefireTests clean test ... Total run:1202 Total errors:0 Total failures:0 ... Ran 457 tests in 16.645s ... [INFO] Ambari Server .. SUCCESS [01:35 min] [INFO] Ambari Agent ... SUCCESS [ 20.611 s] ``` Thanks, Attila Doroszlai
Re: Review Request 62688: AMBARI-22092. Blueprint cluster creation constantly throwing exceptions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62688/ --- (Updated Sept. 29, 2017, 4:30 p.m.) Review request for Ambari, Attila Magyar, Balázs Bence Sári, Jonathan Hurley, and Sandor Magyari. Changes --- fixed 3 unit tests Bugs: AMBARI-22092 https://issues.apache.org/jira/browse/AMBARI-22092 Repository: ambari Description --- Improved logging in `AsyncCallableService`: * Add task name to all log messages, since it is a generic service (although currently only used for a single task). * Introduced new "marker" exception to indicate that task should be retried silently, without logging the exception. * `TimeoutException` is not an error, no need to log with stack trace, timeout is logged anyway. * For other exceptions peel off the `ExecutionException` that the executor framework wraps the actual exception in to reduce verboseness. Improved host group-related messages: * Warn if a host group is completely missing. * Log completion of each host group only once. * Log host groups still waiting for more hosts only once for each host count (eg. log when waiting for 3 hosts, then only when waiting for 2 more hosts). Minor bug fixes: * `AsyncCallableService.timeout` is the total time allowed for repeated task execution. Hence, timeout for subsequent attempts (of `future.get()`) needs to be reduced by the time passed since the task's initial attempt. * `verify()` is a no-op (expects mocks as arg), `verifyAll()` was intended to be called. Moved timeout config logic to `ConfigureClusterTask`. Misc code cleanup. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/topology/AsyncCallableService.java 9a68ea7cd9ee62250fde33ceb0cf2b65b57ae883 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 9769fae4b851d8bdf908c2917d7af57ff0ea061e ambari-server/src/main/java/org/apache/ambari/server/topology/tasks/ConfigureClusterTask.java 60eaa591b9ac3b3ffa417d5c5d84144d4e68d9d1 ambari-server/src/test/java/org/apache/ambari/server/topology/AsyncCallableServiceTest.java 4d962949400485f4d846d87ad74059e0c7eeb9df ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java 0daa20fc5e8d1a55818840407aff9ebd1703cf5c ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java bbf4fdbf01b6498c897093c0efd7a0957e041975 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java 059a8be735e193fe523a32df945a7566dfc22dc7 ambari-server/src/test/java/org/apache/ambari/server/topology/ConfigureClusterTaskTest.java b2dac8f01a817a1f9b0ee6f02a976cdd3304fd94 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java ac643d7c661a1bcb8a0400bff94aeb257aec4101 Diff: https://reviews.apache.org/r/62688/diff/2/ Changes: https://reviews.apache.org/r/62688/diff/1-2/ Testing --- * Manually on cluster with Ambari 2.6: successful completion, successful on retries and timeout. * Unit tests (modified ones are OK; result of the rest is pending) Thanks, Attila Doroszlai
Re: Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 9473cfcd0c7f23ca5b5e63be58bff28cae0a09b1 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AggregateSource.java 64c983f8f8600ee6388769dfc4f1a158c6edff60 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java 665430dacf342aa4343c3a6e4e0a8ab219640d88 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 8dd78cfcdd35cc1c131f396a37177460f6785732 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java 7b0107be9710f9ca2fa0c6e3e246432cabb5a7d1 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AmsSource.java d586f8181be241855e3119924dc8a348724a9497 ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java 11eee05629899e4c1885c568f37d7b091fcec6d6 ambari-server/src/main/java/org/apache/ambari/server/state/alert/ParameterizedSource.java 8ae8d1fb12482c998af3826b53c10f673bd1816f ambari-server/src/main/java/org/apache/ambari/server/state/alert/PercentSource.java 927d8934eae34c7415097234029248b31a6cd908 ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java 7b893824719471b5d356fa1a0be52ebbead36a90 ambari-server/src/main/java/org/apache/ambari/server/state/alert/Reporting.java dc03e2fcaf2aa9bd75563833b73bb5b11ad2a242 ambari-server/src/main/java/org/apache/ambari/server/state/alert/ScriptSource.java a5ed440bc8fc5d68afc0889e3c36fb09d4e9e654 ambari-server/src/main/java/org/apache/ambari/server/state/alert/ServerSource.java c58867a67fee07a8061be37401aacc60cf49e06e ambari-server/src/main/java/org/apache/ambari/server/state/alert/Source.java aa841d59492fed198644d80cb5cefd02307e1f0b ambari-server/src/main/java/org/apache/ambari/server/state/alert/WebSource.java ef946047be538d4b010bd9de6e8e4a48501756ff ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 5141a4b62ba455499c30f69fd11c0da86cd03369 ambari-server/src/test/java/org/apache/ambari/server/agent/AgentSessionManagerTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleanerTest.java 663934fd3ad63b7298a23fd5ab25ab26bc1b879e ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java 50854ce9891ec3b9d31457866965abb8c367f5fd Diff: https://reviews.apache.org/r/61886/diff/6/ Changes: https://reviews.apache.org/r/61886/diff/5-6/ Testing --- * Manually tested server/agent restarts, alert definition changes * New unit test for `AgentSessionManager` Thanks, Attila Doroszlai
Review Request 62688: AMBARI-22092. Blueprint cluster creation constantly throwing exceptions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62688/ --- Review request for Ambari, Attila Magyar, Balázs Bence Sári, Jonathan Hurley, and Sandor Magyari. Bugs: AMBARI-22092 https://issues.apache.org/jira/browse/AMBARI-22092 Repository: ambari Description --- Improved logging in `AsyncCallableService`: * Add task name to all log messages, since it is a generic service (although currently only used for a single task). * Introduced new "marker" exception to indicate that task should be retried silently, without logging the exception. * `TimeoutException` is not an error, no need to log with stack trace, timeout is logged anyway. * For other exceptions peel off the `ExecutionException` that the executor framework wraps the actual exception in to reduce verboseness. Improved host group-related messages: * Warn if a host group is completely missing. * Log completion of each host group only once. * Log host groups still waiting for more hosts only once for each host count (eg. log when waiting for 3 hosts, then only when waiting for 2 more hosts). Minor bug fixes: * `AsyncCallableService.timeout` is the total time allowed for repeated task execution. Hence, timeout for subsequent attempts (of `future.get()`) needs to be reduced by the time passed since the task's initial attempt. * `verify()` is a no-op (expects mocks as arg), `verifyAll()` was intended to be called. Moved timeout config logic to `ConfigureClusterTask`. Misc code cleanup. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/AsyncCallableService.java 9a68ea7cd9ee62250fde33ceb0cf2b65b57ae883 ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java 9769fae4b851d8bdf908c2917d7af57ff0ea061e ambari-server/src/main/java/org/apache/ambari/server/topology/tasks/ConfigureClusterTask.java 60eaa591b9ac3b3ffa417d5c5d84144d4e68d9d1 ambari-server/src/test/java/org/apache/ambari/server/topology/AsyncCallableServiceTest.java 4d962949400485f4d846d87ad74059e0c7eeb9df ambari-server/src/test/java/org/apache/ambari/server/topology/ConfigureClusterTaskTest.java b2dac8f01a817a1f9b0ee6f02a976cdd3304fd94 ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java ac643d7c661a1bcb8a0400bff94aeb257aec4101 Diff: https://reviews.apache.org/r/62688/diff/1/ Testing --- * Manually on cluster with Ambari 2.6: successful completion, successful on retries and timeout. * Unit tests (modified ones are OK; result of the rest is pending) Thanks, Attila Doroszlai
Re: Review Request 62674: A couple of merge fixes for branch-3.0-perf
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62674/#review186591 --- Ship it! Ship It! - Attila Doroszlai On Sept. 28, 2017, 8:13 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62674/ > --- > > (Updated Sept. 28, 2017, 8:13 p.m.) > > > Review request for Ambari and Attila Doroszlai. > > > Bugs: AMBARI-22088 > https://issues.apache.org/jira/browse/AMBARI-22088 > > > Repository: ambari > > > Description > --- > > . > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py > 9f3e9a27a8 > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java > ea208e838e > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > 5f4ba177e2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > d14bd33768 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > e0df48778b > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java > 5d75b9921c > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java > 447d7e8fd4 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java > c97f4cfc6f > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > e6da65e745 > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java > 8743f63aeb > > ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java > ecb616a6b9 > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/templates/input.config-zeppelin.json.j2 > 2b373d54e5 > ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml > 4a89fd4b7b > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java > 439a5542c9 > > ambari-server/src/test/java/org/apache/ambari/server/agent/DummyHeartbeatConstants.java > 69980dc6eb > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java > f55db0831b > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java > 4b4da49762 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java > 1b37435a89 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > 883e8918b6 > > ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListenerTest.java > ff6fab9e3b > > ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java > 677b607834 > > ambari-server/src/test/java/org/apache/ambari/server/state/alerts/InitialAlertEventTest.java > 3d25ef718e > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > 95a04deb6b > > > Diff: https://reviews.apache.org/r/62674/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 62584: Remove Database auto configuration from Infra Manager
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62584/#review186307
---
Ship it!
Ship It!
- Attila Doroszlai
On Sept. 26, 2017, 8:30 p.m., Oliver Szabo wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62584/
> ---
>
> (Updated Sept. 26, 2017, 8:30 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Laszlo Puskas, Miklos Gergely,
> Robert Nettleton, and Sandor Magyari.
>
>
> Bugs: AMBARI-22064
> https://issues.apache.org/jira/browse/AMBARI-22064
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Spring boot introduced a Database autoconfiguration feature. Becuase of that
> feature, our initialization script does not work.
>
> To use that properly we need to set some spring properties
> ("spring.datasource" prefix) which can use the right database schema script
> with discovering the datasorce type of the datasource (which sounds fine),
> but we should not use that feature until we find out how to change the
> default prefixes of the configuration.
>
>
> Diffs
> -
>
>
> ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/InfraManager.java
> 185e344
>
>
> Diff: https://reviews.apache.org/r/62584/diff/1/
>
>
> Testing
> ---
>
> manually, db file created properly
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 62563: Reduce alert reports performance overhead
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62563/#review186227 --- Fix it, then Ship it! Server-side change for this enhancement is also addressed in https://reviews.apache.org/r/61886/ ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java Line 170 (original), 174 (patched) <https://reviews.apache.org/r/62563/#comment262713> This null check should precede usage of `definition`. - Attila Doroszlai On Sept. 26, 2017, 12:05 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62563/ > --- > > (Updated Sept. 26, 2017, 12:05 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22057 > https://issues.apache.org/jira/browse/AMBARI-22057 > > > Repository: ambari > > > Description > --- > > 1\. Send alerts only if status or text changed. This reduces bandwidth usage > and number of operations (DB and others) done by server. > 2\. Don't send some redundant fields in alert reports. This reduces bandwidth > usage. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/AlertStatusReporter.py 3d5a62e > ambari-agent/src/main/python/ambari_agent/ClusterCache.py 7704443 > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py e364447 > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java > 3607df4 > > > Diff: https://reviews.apache.org/r/62563/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
3607df4f0c9c96ee0bed741a12195a4d428e6b90 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/requests/StateUpdateListener.java 27af7178c497f8ac171b4c788a966cfcb6de9278 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 25bc8136510574cc54fa2f3365c755e6a9426547 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AggregateSource.java 46ddd355b67e75da9cf98b365d830bd8cd7e20bd ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java e197cd0050c97fa655ceefa21a5763227c1890ec ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionHash.java 76905085aab60771474fac4979ccbe560245aa9b ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java c2ef5e7859464c22ce64a90a6c771eeff803ba38 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AmsSource.java 21cf81c8cd0abd91940b73e60c40db4a68496637 ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java 25734cd496a3d3fa2f7f89716c1080ca2798c069 ambari-server/src/main/java/org/apache/ambari/server/state/alert/ParameterizedSource.java 92c49203a44bfd5d829f7f1835e97116029f1ee6 ambari-server/src/main/java/org/apache/ambari/server/state/alert/PercentSource.java 57bba06d723470e6f8c26a4961c7e37db05ebdfa ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java e972c621d7361538105cb54734cb2c0b582daef8 ambari-server/src/main/java/org/apache/ambari/server/state/alert/Reporting.java 4aaefab33713c7641cab3775c615e87c1d96d82c ambari-server/src/main/java/org/apache/ambari/server/state/alert/ScriptSource.java d1b70701ae26c3117f7b9fb8ece2676c0f3d45cb ambari-server/src/main/java/org/apache/ambari/server/state/alert/ServerSource.java d5a69abcbb36bebd2776e61f5aa3ebcc3724df63 ambari-server/src/main/java/org/apache/ambari/server/state/alert/Source.java 76054a9c8cb3e0f79ce50c2b8be2699f0dd6741f ambari-server/src/main/java/org/apache/ambari/server/state/alert/WebSource.java a3c0ada908716e59db8516a8bdef75b7c2814806 ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 446ae8c4850eb60dcc0d18f93a4b3ddbd4882a35 ambari-server/src/test/java/org/apache/ambari/server/agent/AgentSessionManagerTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java c0d445a32f6a8153ea19ffc4cb25874637929fb5 Diff: https://reviews.apache.org/r/61886/diff/5/ Changes: https://reviews.apache.org/r/61886/diff/4-5/ Testing (updated) --- * Manually tested server/agent restarts, alert definition changes * New unit test for `AgentSessionManager` Thanks, Attila Doroszlai
Re: Review Request 62465: AMBARI-22025. Service auto start broken due to incomplete execution command
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62465/ --- (Updated Sept. 21, 2017, 8:29 p.m.) Review request for Ambari, Balázs Bence Sári, Jonathan Hurley, and Nate Cole. Changes --- unit test coverage Bugs: AMBARI-22025 https://issues.apache.org/jira/browse/AMBARI-22025 Repository: ambari Description --- Execution commands embedded in status commands are created programmatically instead of deserializing JSON. AMBARI-21361 centralized addition of `service_package_folder` and `hooks_folder`, but the logic is not executed in this case. Fix: reuse the logic (extracted to the method `setVersions`) which adds `service_package_folder` and a few other parameters. (patch is for branch-2.6) Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java 7b202598d5bb88d78dc8486ed2801f87efb326a8 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 96ed2613888c1badc50c21f540868444e5698b51 ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java 61e322f71a373c0e3c69d44b89ef0b270aa6a85c Diff: https://reviews.apache.org/r/62465/diff/2/ Changes: https://reviews.apache.org/r/62465/diff/1-2/ Testing --- * Tested manually on local cluster per STR. * Unit tests: `Tests run: 4909, Failures: 0, Errors: 0, Skipped: 34` Thanks, Attila Doroszlai
Review Request 62465: AMBARI-22025. Service auto start broken due to incomplete execution command
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62465/ --- Review request for Ambari, Balázs Bence Sári, Jonathan Hurley, and Nate Cole. Bugs: AMBARI-22025 https://issues.apache.org/jira/browse/AMBARI-22025 Repository: ambari Description --- Execution commands embedded in status commands are created programmatically instead of deserializing JSON. AMBARI-21361 centralized addition of `service_package_folder` and `hooks_folder`, but the logic is not executed in this case. Fix: reuse the logic (extracted to the method `setVersions`) which adds `service_package_folder` and a few other parameters. (patch is for branch-2.6) Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java 7b202598d5bb88d78dc8486ed2801f87efb326a8 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 96ed2613888c1badc50c21f540868444e5698b51 Diff: https://reviews.apache.org/r/62465/diff/1/ Testing --- * Tested manually on local cluster per STR. * Unit tests: `Tests run: 4909, Failures: 0, Errors: 0, Skipped: 34` Thanks, Attila Doroszlai
Re: Review Request 62402: Multiple fixes into branch-3.0-perf
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62402/#review185670 --- Ship it! Ship It! - Attila Doroszlai On Sept. 19, 2017, 10:53 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62402/ > --- > > (Updated Sept. 19, 2017, 10:53 a.m.) > > > Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Vitalyi > Brodetskyi. > > > Bugs: AMBARI-21989 > https://issues.apache.org/jira/browse/AMBARI-21989 > > > Repository: ambari > > > Description > --- > > * restart triggered by server should gracefully finish operations > * if status command fails, exception should be logged. > * move configs logic from InitializerModule into AmbariConfig, where it > belongs > * fix typo (alert_def*E*nitions into alert_def*I*nitions) > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py 94e72c2 > ambari-agent/src/main/python/ambari_agent/AlertStatusReporter.py 2bd2383 > ambari-agent/src/main/python/ambari_agent/AmbariConfig.py e1c40c8 > ambari-agent/src/main/python/ambari_agent/CommandStatusDict.py c681550 > ambari-agent/src/main/python/ambari_agent/CommandStatusReporter.py 652574f > ambari-agent/src/main/python/ambari_agent/ComponentStatusExecutor.py > f85ba42 > ambari-agent/src/main/python/ambari_agent/Constants.py 6a258d2 > ambari-agent/src/main/python/ambari_agent/FileCache.py 0e598e9 > ambari-agent/src/main/python/ambari_agent/HeartbeatThread.py b6e1aaf > ambari-agent/src/main/python/ambari_agent/HostStatusReporter.py 09ce016 > ambari-agent/src/main/python/ambari_agent/InitializerModule.py 0126250 > ambari-agent/src/main/python/ambari_agent/Utils.py af70d77 > > ambari-agent/src/main/python/ambari_agent/listeners/AlertDefinitionsEventListener.py > 91ae5de > ambari-agent/src/main/python/ambari_agent/main.py 05fc5ce > ambari-agent/src/test/python/ambari_agent/TestAgentStompResponses.py > 3c571ab > > > Diff: https://reviews.apache.org/r/62402/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 62379: Fixes to CommandStatusDict
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62379/#review185547 --- Ship it! Ship It! - Attila Doroszlai On Sept. 18, 2017, 12:56 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62379/ > --- > > (Updated Sept. 18, 2017, 12:56 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-21979 > https://issues.apache.org/jira/browse/AMBARI-21979 > > > Repository: ambari > > > Description > --- > > 1\. Fix COMPLETE status for command sent twice > 2\. Fix possible race conditions > 3\. Add setting to avoid periodically reporting stdout/stderr. Can be used on > perf clusters to reduce bandwidth usage for perf. Previously to do this > reporting thread had to be stopped which caused issues with delayed commands > status re-sending. > > > Diffs > - > > ambari-agent/conf/unix/ambari-agent.ini e1df1d2 > ambari-agent/src/main/python/ambari_agent/AlertStatusReporter.py 084a342 > ambari-agent/src/main/python/ambari_agent/CommandStatusDict.py fa71d15 > ambari-agent/src/main/python/ambari_agent/CommandStatusReporter.py f46c47a > ambari-agent/src/main/python/ambari_agent/ComponentStatusExecutor.py > d074786c > ambari-agent/src/main/python/ambari_agent/Constants.py e36eda5 > ambari-agent/src/main/python/ambari_agent/InitializerModule.py 4fbef65 > > > Diff: https://reviews.apache.org/r/62379/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 62081: AMBARI-21878. Implement STOMP endpoint to handle alerts from agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62081/ --- (Updated Sept. 7, 2017, 2:02 p.m.) Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, Sandor Magyari, Sebastian Toader, and Sid Wagle. Changes --- removed `Alert.cluster` (cluster name) in favor of `clusterId` Bugs: AMBARI-21878 https://issues.apache.org/jira/browse/AMBARI-21878 Repository: ambari Description --- * Process alerts from agent at `/reports/alerts_status` * Add `clusterId` to alert reports since cluster name is empty Diffs (updated) - ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 76d83908883a1ad1c5ee44534592e86e6b75451a ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java fce6be6a92104d775e5105c2056b05ff89abcfdf ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentReportsController.java 0152d89c80d5ef63850d3a9211ca7a6c7bcb72b5 ambari-server/src/main/java/org/apache/ambari/server/alerts/AgentHeartbeatAlertRunnable.java 41be01e12f596ec6229ccbbc44e54f93827a2328 ambari-server/src/main/java/org/apache/ambari/server/alerts/AmbariPerformanceRunnable.java a35e6fd44a134658284113725dfd8706e470353c ambari-server/src/main/java/org/apache/ambari/server/alerts/StaleAlertRunnable.java 503ad9281feb82a4523cb4f5ed7e9b74740d7d13 ambari-server/src/main/java/org/apache/ambari/server/events/AlertReceivedEvent.java b17071c7f384dba1cb508066fa5f808ada7487e4 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertAggregateListener.java 1ed19a4e46d2008f0aba05b19bdbdc160b0b3437 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 23a547fbdd7e1b1fb8b6c6e660daade1e51cb4e1 ambari-server/src/main/java/org/apache/ambari/server/state/Alert.java 1caa0e3221ac1452aa4822ad78e690f737dc878a ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java aa53564661c49f9198d0c89ba14595258af8259b ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java aecfabe52abf448fc8782992f1dd85c2767a3cc7 ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java 9a6391ca33f783847ccd79352d196515be7737b3 ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java 68395b5cb7d23a6b37b51660aec2aeb8d20e7130 ambari-server/src/test/java/org/apache/ambari/server/state/alerts/InitialAlertEventTest.java 890464da368b658dfb5448419917699c8e1f9c6d ambari-server/src/test/java/org/apache/ambari/server/state/cluster/AlertDataManagerTest.java 8d7d8ad85dcc1ec97d10b089976b5d9f20549cb7 Diff: https://reviews.apache.org/r/62081/diff/2/ Changes: https://reviews.apache.org/r/62081/diff/1-2/ Testing --- Manually tested on local cluster. Thanks, Attila Doroszlai
Re: Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
/java/org/apache/ambari/server/state/alert/Source.java 76054a9c8cb3e0f79ce50c2b8be2699f0dd6741f ambari-server/src/main/java/org/apache/ambari/server/state/alert/WebSource.java a3c0ada908716e59db8516a8bdef75b7c2814806 Diff: https://reviews.apache.org/r/61886/diff/4/ Changes: https://reviews.apache.org/r/61886/diff/3-4/ Testing --- Manually tested server/agent restarts, alert definition changes. Thanks, Attila Doroszlai
Re: Review Request 62151: Component status reports are sent for the components which are not on the host
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62151/#review184803 --- Ship it! Ship It! - Attila Doroszlai On Sept. 7, 2017, 10:34 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62151/ > --- > > (Updated Sept. 7, 2017, 10:34 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-21899 > https://issues.apache.org/jira/browse/AMBARI-21899 > > > Repository: ambari > > > Description > --- > > . > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/ClusterTopologyCache.py e35b219 > ambari-agent/src/main/python/ambari_agent/ComponentStatusExecutor.py > 4ccd815 > > > Diff: https://reviews.apache.org/r/62151/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 62084: Throw an error if unsupported database JDBC driver is configured for HDP services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62084/#review184637 --- Ship it! Ship It! - Attila Doroszlai On Sept. 6, 2017, 8:41 a.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62084/ > --- > > (Updated Sept. 6, 2017, 8:41 a.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, and Robert Levas. > > > Bugs: AMBARI-21882 > https://issues.apache.org/jira/browse/AMBARI-21882 > > > Repository: ambari > > > Description > --- > > Raise an exception in case an unsupported db type/jdbc driver is specified in > HDP service configs. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py > c80c577 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py > 39c06f2 > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py > 90d9067 > > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py > c31181d > > ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/params.py > 24f459c > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py > 466646b > > ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/params.py > 003eee1 > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py > c1138b3 > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/params_linux.py > c1138b3 > ambari-server/src/main/resources/custom_actions/scripts/check_host.py > a485415 > > ambari-server/src/test/python/common-services/HIVE/test_jdbc_driver_config.py > PRE-CREATION > > ambari-server/src/test/python/common-services/RANGER/test_db_flavor_config.py > PRE-CREATION > > ambari-server/src/test/python/common-services/RANGER_KMS/test_db_flavor_config.py > PRE-CREATION > > ambari-server/src/test/python/common-services/SQOOP/test_jdbc_driver_config.py > PRE-CREATION > ambari-server/src/test/python/common-services/configs/hive_default.json > PRE-CREATION > > ambari-server/src/test/python/common-services/configs/hive_unsupported_jdbc_type.json > PRE-CREATION > > ambari-server/src/test/python/common-services/configs/ranger_admin_default.json > PRE-CREATION > > ambari-server/src/test/python/common-services/configs/ranger_admin_unsupported_db_flavor.json > PRE-CREATION > > ambari-server/src/test/python/common-services/configs/ranger_kms_default.json > PRE-CREATION > > ambari-server/src/test/python/common-services/configs/ranger_kms_unsupported_db_flavor.json > PRE-CREATION > ambari-server/src/test/python/common-services/configs/sqoop_default.json > PRE-CREATION > > ambari-server/src/test/python/common-services/configs/sqoop_unsupported_jdbc_driver.json > PRE-CREATION > ambari-server/src/test/python/custom_actions/TestCheckHost.py 0d4e485 > > > Diff: https://reviews.apache.org/r/62084/diff/1/ > > > Testing > --- > > ** mvn -pl ambari-server -DskipSurefireTests test ** > ``` > -- > Ran 253 tests in 6.889s > > OK > -- > Failed tests: > FAIL: test_start_secured (test_zeppelin_master.TestZeppelinMaster) > -- > Traceback (most recent call last): > File > "/Users/stoader/Projects/Hortonworks/apache/ambari/ambari-common/src/test/python/mock/mock.py", > line 1199, in patched > return func(*args, **keywargs) > File > "/Users/stoader/Projects/Hortonworks/apache/ambari/ambari-server/src/test/python/stacks/2.5/ZEPPELIN/test_zeppelin_master.py", > line 324, in test_start_secured > group = 'zeppelin', > File > "/Users/stoader/Projects/Hortonworks/apache/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py", > line 328, in assertResourceCalled > self.assertEquals(resource_type, resource.__class__.__name__) > AssertionError: 'File' != 'Execute' > > -- > Total run:1194 > Total errors:0 > Total failures:1 > > ``` > > The one failing test ```test_zeppelin_master.py``` is unrelated to this > change. > > > Thanks, > > Sebastian Toader > >
Re: Review Request 62012: Agent sometimes hangs during connection to server
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62012/#review184631 --- Ship it! Ship It! - Attila Doroszlai On Sept. 1, 2017, 8:42 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62012/ > --- > > (Updated Sept. 1, 2017, 8:42 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Sid > Wagle, and Vitalyi Brodetskyi. > > > Bugs: AMBARI-21861 > https://issues.apache.org/jira/browse/AMBARI-21861 > > > Repository: ambari > > > Description > --- > > At 5k we saw this a couple of times. Agent hangs during do_handshake while > establishing ssl connection. > > > > # ThreadID: 139752294471424 > File: "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap > self.__bootstrap_inner() > File: "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner > self.run() > File: "/usr/lib/python2.6/site-packages/ambari_agent/HeartbeatThread.py", > line 89, in run > self.register() > File: "/usr/lib/python2.6/site-packages/ambari_agent/HeartbeatThread.py", > line 120, in register > self.establish_connection() > File: "/usr/lib/python2.6/site-packages/ambari_agent/HeartbeatThread.py", > line 214, in establish_connection > self.connection = security.establish_connection(connection_url) > File: "/usr/lib/python2.6/site-packages/ambari_agent/security.py", line > 114, in establish_connection > conn.start() > File: "/usr/lib/python2.6/site-packages/ambari_stomp/connect.py", line > 46, in start > self.transport.start() > File: "/usr/lib/python2.6/site-packages/ambari_stomp/transport.py", line > 109, in start > self.attempt_connection() > File: > "/usr/lib/python2.6/site-packages/ambari_stomp/adapter/websocket.py", line > 89, in attempt_connection > self.ws.connect() > File: "/usr/lib/python2.6/site-packages/ambari_ws4py/client/__init__.py", > line 215, in connect > self.sock.connect(self.bind_addr) > File: "/usr/lib64/python2.6/ssl.py", line 295, in connect > self.do_handshake() > File: "/usr/lib64/python2.6/ssl.py", line 279, in do_handshake > self._sslobj.do_handshake() > > > Diffs > - > > ambari-common/src/main/python/ambari_ws4py/client/__init__.py 89598ab > > > Diff: https://reviews.apache.org/r/62012/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61886/ --- (Updated Sept. 5, 2017, 8:20 p.m.) Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, Sandor Magyari, Sebastian Toader, and Sid Wagle. Changes --- added @JsonProperty where necessary Bugs: AMBARI-21803 https://issues.apache.org/jira/browse/AMBARI-21803 Repository: ambari Description --- * Added endpoint * Created data structure to match expected JSON format * Extracted some common logic to parent classes Diffs (updated) - ambari-agent/src/main/python/ambari_agent/ClusterAlertDefinitionsCache.py d3ab48e4da79cc7615bd5dedfec3754ffe5accc2 ambari-agent/src/main/python/ambari_agent/Constants.py e36eda577d0195055ebe89e3eeac5c63aa4a460b ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentClusterDataHolder.java 0f0a4919ba7e1343c31c8397d5309542c2463287 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentConfigsHolder.java 0de686dfaad7a3c437fa5f9af5c1fa8abd4e61e8 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentCurrentDataController.java 7035f38cfbb7a75bd4cb5721a542ef832a805c99 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentDataHolder.java 635611bf6d325deb33cde671556647b9c264a716 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentHostDataHolder.java 792f10ef33363fd57579792f51901681ad83aa0b ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AlertDefinitionsHolder.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/HostLevelParamsHolder.java db093f85a7f304e895bad098a3ca5a62d7a3859d ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/MetadataHolder.java b7e680627f3fd42e87b7152f9ab9de29f63a44fa ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/TopologyHolder.java 0da5d831d546da98e2284b048b914f972cacfc05 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertCluster.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/MetadataCluster.java 3ee5ad6d00b4c4374cba48e80d1cb410500ee586 ambari-server/src/main/java/org/apache/ambari/server/events/AgentConfigsUpdateEvent.java 640be2d89bce55a9f58cbaf6b4e5296df9fc1a8c ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionUpdateHolder.java e820401d460bc672596a4ede60abcd39646be49e ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionsUpdateEvent.java cbe6e3703960ce0af9563f41b63d2a71427b4267 ambari-server/src/main/java/org/apache/ambari/server/events/HostLevelParamsUpdateEvent.java 66ab38ea39bea2c100a2245ad422a4c6a037d7ed ambari-server/src/main/java/org/apache/ambari/server/events/MetadataUpdateEvent.java 239b5b8184a02a057c2dc9d4a53c671dc06d9806 ambari-server/src/main/java/org/apache/ambari/server/events/TopologyAgentUpdateEvent.java 1fa4e6c20d0b2d31e376b7b8e563326a7cc1e06a ambari-server/src/main/java/org/apache/ambari/server/events/TopologyUpdateEvent.java 1b5b90b562babe4cea18a1bd93104d72922d31fa ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 25bc8136510574cc54fa2f3365c755e6a9426547 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AggregateSource.java 46ddd355b67e75da9cf98b365d830bd8cd7e20bd ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java e197cd0050c97fa655ceefa21a5763227c1890ec ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertUri.java c2ef5e7859464c22ce64a90a6c771eeff803ba38 ambari-server/src/main/java/org/apache/ambari/server/state/alert/AmsSource.java 21cf81c8cd0abd91940b73e60c40db4a68496637 ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java 25734cd496a3d3fa2f7f89716c1080ca2798c069 ambari-server/src/main/java/org/apache/ambari/server/state/alert/ParameterizedSource.java 92c49203a44bfd5d829f7f1835e97116029f1ee6 ambari-server/src/main/java/org/apache/ambari/server/state/alert/PercentSource.java 57bba06d723470e6f8c26a4961c7e37db05ebdfa ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java e972c621d7361538105cb54734cb2c0b582daef8 Diff: https://reviews.apache.org/r/61886/diff/3/ Changes: https://reviews.apache.org/r/61886/diff/2-3/ Testing --- Manually tested server/agent restarts, alert definition changes. Thanks, Attila Doroszlai
Re: Review Request 62081: AMBARI-21878. Implement STOMP endpoint to handle alerts from agent
> On Sept. 5, 2017, 2:44 p.m., Andrew Onischuk wrote: > > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py > > Line 149 (original), 149 (patched) > > <https://reviews.apache.org/r/62081/diff/1/?file=1814401#file1814401line149> > > > > Can we remove this line? Sure. Should I also remove cluster name from server-side alert code? - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62081/#review184532 --- On Sept. 5, 2017, 2:43 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62081/ > --- > > (Updated Sept. 5, 2017, 2:43 p.m.) > > > Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, Sandor > Magyari, Sebastian Toader, and Sid Wagle. > > > Bugs: AMBARI-21878 > https://issues.apache.org/jira/browse/AMBARI-21878 > > > Repository: ambari > > > Description > --- > > * Process alerts from agent at `/reports/alerts_status` > * Add `clusterId` to alert reports since cluster name is empty > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py > 76d83908883a1ad1c5ee44534592e86e6b75451a > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java > fce6be6a92104d775e5105c2056b05ff89abcfdf > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentReportsController.java > 0152d89c80d5ef63850d3a9211ca7a6c7bcb72b5 > > ambari-server/src/main/java/org/apache/ambari/server/events/AlertReceivedEvent.java > b17071c7f384dba1cb508066fa5f808ada7487e4 > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java > 23a547fbdd7e1b1fb8b6c6e660daade1e51cb4e1 > ambari-server/src/main/java/org/apache/ambari/server/state/Alert.java > 1caa0e3221ac1452aa4822ad78e690f737dc878a > > > Diff: https://reviews.apache.org/r/62081/diff/1/ > > > Testing > --- > > Manually tested on local cluster. > > > Thanks, > > Attila Doroszlai > >
Review Request 62081: AMBARI-21878. Implement STOMP endpoint to handle alerts from agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62081/ --- Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, Sandor Magyari, Sebastian Toader, and Sid Wagle. Bugs: AMBARI-21878 https://issues.apache.org/jira/browse/AMBARI-21878 Repository: ambari Description --- * Process alerts from agent at `/reports/alerts_status` * Add `clusterId` to alert reports since cluster name is empty Diffs - ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 76d83908883a1ad1c5ee44534592e86e6b75451a ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java fce6be6a92104d775e5105c2056b05ff89abcfdf ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentReportsController.java 0152d89c80d5ef63850d3a9211ca7a6c7bcb72b5 ambari-server/src/main/java/org/apache/ambari/server/events/AlertReceivedEvent.java b17071c7f384dba1cb508066fa5f808ada7487e4 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 23a547fbdd7e1b1fb8b6c6e660daade1e51cb4e1 ambari-server/src/main/java/org/apache/ambari/server/state/Alert.java 1caa0e3221ac1452aa4822ad78e690f737dc878a Diff: https://reviews.apache.org/r/62081/diff/1/ Testing --- Manually tested on local cluster. Thanks, Attila Doroszlai
Re: Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61886/ --- (Updated Sept. 5, 2017, 2:39 p.m.) Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, Sandor Magyari, Sebastian Toader, and Sid Wagle. Changes --- minor tweak Bugs: AMBARI-21803 https://issues.apache.org/jira/browse/AMBARI-21803 Repository: ambari Description --- * Added endpoint * Created data structure to match expected JSON format * Extracted some common logic to parent classes Diffs (updated) - ambari-agent/src/main/python/ambari_agent/ClusterAlertDefinitionsCache.py d3ab48e4da79cc7615bd5dedfec3754ffe5accc2 ambari-agent/src/main/python/ambari_agent/Constants.py e36eda577d0195055ebe89e3eeac5c63aa4a460b ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentClusterDataHolder.java 0f0a4919ba7e1343c31c8397d5309542c2463287 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentConfigsHolder.java 0de686dfaad7a3c437fa5f9af5c1fa8abd4e61e8 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentCurrentDataController.java 7035f38cfbb7a75bd4cb5721a542ef832a805c99 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentDataHolder.java 635611bf6d325deb33cde671556647b9c264a716 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentHostDataHolder.java 792f10ef33363fd57579792f51901681ad83aa0b ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AlertDefinitionsHolder.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/HostLevelParamsHolder.java db093f85a7f304e895bad098a3ca5a62d7a3859d ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/MetadataHolder.java b7e680627f3fd42e87b7152f9ab9de29f63a44fa ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/TopologyHolder.java 0da5d831d546da98e2284b048b914f972cacfc05 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertCluster.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/MetadataCluster.java 3ee5ad6d00b4c4374cba48e80d1cb410500ee586 ambari-server/src/main/java/org/apache/ambari/server/events/AgentConfigsUpdateEvent.java 640be2d89bce55a9f58cbaf6b4e5296df9fc1a8c ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionUpdateHolder.java e820401d460bc672596a4ede60abcd39646be49e ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionsUpdateEvent.java cbe6e3703960ce0af9563f41b63d2a71427b4267 ambari-server/src/main/java/org/apache/ambari/server/events/HostLevelParamsUpdateEvent.java 66ab38ea39bea2c100a2245ad422a4c6a037d7ed ambari-server/src/main/java/org/apache/ambari/server/events/MetadataUpdateEvent.java 239b5b8184a02a057c2dc9d4a53c671dc06d9806 ambari-server/src/main/java/org/apache/ambari/server/events/TopologyAgentUpdateEvent.java 1fa4e6c20d0b2d31e376b7b8e563326a7cc1e06a ambari-server/src/main/java/org/apache/ambari/server/events/TopologyUpdateEvent.java 1b5b90b562babe4cea18a1bd93104d72922d31fa ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 25bc8136510574cc54fa2f3365c755e6a9426547 Diff: https://reviews.apache.org/r/61886/diff/2/ Changes: https://reviews.apache.org/r/61886/diff/1-2/ Testing --- Manually tested server/agent restarts, alert definition changes. Thanks, Attila Doroszlai
Re: Review Request 52785: Fix problem on Suse for ambari-agent installation
> On Oct. 13, 2016, 2:26 p.m., Attila Doroszlai wrote: > > ambari-server/src/main/python/setupAgent.py > > Line 125 (original), 125 (patched) > > <https://reviews.apache.org/r/52785/diff/1/?file=1532679#file1532679line125> > > > > The `-y` flag for `install` is an alias for `--non-interactive`, so > > this change is not strictly necessary. > > > > https://en.opensuse.org/SDB:Zypper_manual_(plain) > > Zsombor Gegesy wrote: > Interesting. In this case, I'm very curious, why that flag didn't work, > and stucked in the middle, unfortunately I can't reproduce it now. > > Attila Doroszlai wrote: > It was stuck at "zypper search", not this one. > > Zsombor Gegesy wrote: > Sorry, I was confused a bit, obviously. > > Could you merge it? Already merged it earlier today, so the review request can be closed. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52785/#review152483 --- On Oct. 18, 2016, 1:05 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52785/ > ------- > > (Updated Oct. 18, 2016, 1:05 p.m.) > > > Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, and Sandor > Magyari. > > > Bugs: AMBARI-18577 > https://issues.apache.org/jira/browse/AMBARI-18577 > > > Repository: ambari > > > Description > --- > > Add "--non-interactive" flag to zypper invocations in setupAgent > > > Diffs > - > > ambari-server/src/main/python/setupAgent.py 9a1cae9 > > > Diff: https://reviews.apache.org/r/52785/diff/3/ > > > Testing > --- > > I've tested on a Suse 11p3, with manually editing the setupAgent.py file, to > start the installation. > Unfortunately I couldn't build ambari locally, some npm/brunch compilation > error blocks maven > > > Thanks, > > Zsombor Gegesy > >
Re: Review Request 52785: Fix problem on Suse for ambari-agent installation
> On Oct. 13, 2016, 2:26 p.m., Attila Doroszlai wrote: > > ambari-server/src/main/python/setupAgent.py > > Line 125 (original), 125 (patched) > > <https://reviews.apache.org/r/52785/diff/1/?file=1532679#file1532679line125> > > > > The `-y` flag for `install` is an alias for `--non-interactive`, so > > this change is not strictly necessary. > > > > https://en.opensuse.org/SDB:Zypper_manual_(plain) > > Zsombor Gegesy wrote: > Interesting. In this case, I'm very curious, why that flag didn't work, > and stucked in the middle, unfortunately I can't reproduce it now. It was stuck at "zypper search", not this one. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52785/#review152483 --- On Oct. 18, 2016, 1:05 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52785/ > ------- > > (Updated Oct. 18, 2016, 1:05 p.m.) > > > Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, and Sandor > Magyari. > > > Bugs: AMBARI-18577 > https://issues.apache.org/jira/browse/AMBARI-18577 > > > Repository: ambari > > > Description > --- > > Add "--non-interactive" flag to zypper invocations in setupAgent > > > Diffs > - > > ambari-server/src/main/python/setupAgent.py 9a1cae9 > > > Diff: https://reviews.apache.org/r/52785/diff/3/ > > > Testing > --- > > I've tested on a Suse 11p3, with manually editing the setupAgent.py file, to > start the installation. > Unfortunately I couldn't build ambari locally, some npm/brunch compilation > error blocks maven > > > Thanks, > > Zsombor Gegesy > >
Re: Review Request 62064: Support for handling ldap admin group mappings
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62064/#review184512 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java Line 74 (original), 76 (patched) <https://reviews.apache.org/r/62064/#comment260678> Ambari Server codebase seems to prefer `LOG`, not `LOGGER` (310 vs. 20 instances). ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java Lines 795 (patched) <https://reviews.apache.org/r/62064/#comment260679> * Missing message param: 2 placeholders, but only 1 value. * Message says "entry count", but actual value passed is the set, not its size. - Attila Doroszlai On Sept. 4, 2017, 3:47 p.m., Laszlo Puskas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62064/ > --- > > (Updated Sept. 4, 2017, 3:47 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, and Robert Levas. > > > Bugs: AMBARI-21873 > https://issues.apache.org/jira/browse/AMBARI-21873 > > > Repository: ambari > > > Description > --- > > This feature adds the possibility to handle users belonging to a defined LDAP > groups as ambari administrators during the LDAP sync. > The list of the groups that need to be considered is stored in the ambari > property: > ```java > authorization.ldap.adminGroupMappingRules > ``` > > The solution is to grant admin privileges to users belonging to these groups > on LDPA sync. > > Warning: > * changes in the LDAP group memberships will not be reflected in Ambari after > the sync (eg.: administrator privileges won't be automatically revoked if > users are removed from the groups listed in the property) > * administrator privileges can be granted/removed by another administrator > using the ambari UI, thus these actions can interfere > * if groups are not synced, this property is not taken into account > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java > 7d8c683 > > > Diff: https://reviews.apache.org/r/62064/diff/1/ > > > Testing > --- > > Manually: > > ambari-server sync-ldap —all > - all users made admin > > ambari-server sync-ldap --users /tmp/users.csv > - user imported, property not taken into account > > ambari-server sync-ldap —groups /tmp/groups.csv > - the csv contains a group in the mapping rule, all users in LDAP belonging > to the group imported and made admin > > ambari-server sync-ldap —groups /tmp/groups.csv > - manually modified the admin (revoked admin provileges) > - the ldap sync didn’t override the manual setting > > Unit tests ... in progress > > > Thanks, > > Laszlo Puskas > >
Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61886/ --- Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, and Sid Wagle. Bugs: AMBARI-21803 https://issues.apache.org/jira/browse/AMBARI-21803 Repository: ambari Description --- * Added endpoint * Created data structure to match expected JSON format * Extracted some common logic to parent classes Diffs - ambari-agent/src/main/python/ambari_agent/ClusterAlertDefinitionsCache.py d3ab48e4da79cc7615bd5dedfec3754ffe5accc2 ambari-agent/src/main/python/ambari_agent/Constants.py e36eda577d0195055ebe89e3eeac5c63aa4a460b ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentClusterDataHolder.java 0f0a4919ba7e1343c31c8397d5309542c2463287 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentConfigsHolder.java 0de686dfaad7a3c437fa5f9af5c1fa8abd4e61e8 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentCurrentDataController.java 7035f38cfbb7a75bd4cb5721a542ef832a805c99 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentDataHolder.java 635611bf6d325deb33cde671556647b9c264a716 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentHostDataHolder.java 792f10ef33363fd57579792f51901681ad83aa0b ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AlertDefinitionsHolder.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/HostLevelParamsHolder.java db093f85a7f304e895bad098a3ca5a62d7a3859d ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/MetadataHolder.java b7e680627f3fd42e87b7152f9ab9de29f63a44fa ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/TopologyHolder.java 0da5d831d546da98e2284b048b914f972cacfc05 ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertCluster.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/MetadataCluster.java 3ee5ad6d00b4c4374cba48e80d1cb410500ee586 ambari-server/src/main/java/org/apache/ambari/server/events/AgentConfigsUpdateEvent.java 640be2d89bce55a9f58cbaf6b4e5296df9fc1a8c ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionUpdateHolder.java e820401d460bc672596a4ede60abcd39646be49e ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionsUpdateEvent.java cbe6e3703960ce0af9563f41b63d2a71427b4267 ambari-server/src/main/java/org/apache/ambari/server/events/HostLevelParamsUpdateEvent.java 66ab38ea39bea2c100a2245ad422a4c6a037d7ed ambari-server/src/main/java/org/apache/ambari/server/events/MetadataUpdateEvent.java 239b5b8184a02a057c2dc9d4a53c671dc06d9806 ambari-server/src/main/java/org/apache/ambari/server/events/TopologyAgentUpdateEvent.java 1fa4e6c20d0b2d31e376b7b8e563326a7cc1e06a ambari-server/src/main/java/org/apache/ambari/server/events/TopologyUpdateEvent.java 1b5b90b562babe4cea18a1bd93104d72922d31fa ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 25bc8136510574cc54fa2f3365c755e6a9426547 Diff: https://reviews.apache.org/r/61886/diff/1/ Testing --- Manually tested server/agent restarts, alert definition changes. Thanks, Attila Doroszlai
Re: Review Request 61847: Component version is not reported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61847/#review183733 --- Ship it! Ship It! - Attila Doroszlai On Aug. 23, 2017, 2:23 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61847/ > --- > > (Updated Aug. 23, 2017, 2:23 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sumit > Mohanty. > > > Bugs: AMBARI-21789 > https://issues.apache.org/jira/browse/AMBARI-21789 > > > Repository: ambari > > > Description > --- > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/script/script.py > 269fc72 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 80c3623 > > > Diff: https://reviews.apache.org/r/61847/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 61781: Cache hashes should not be reset after restart ; some ambari-server restart fixes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61781/#review183451 --- Fix it, then Ship it! ambari-agent/src/main/python/ambari_agent/ClusterCache.py Lines 57-62 (original), 58-68 (patched) <https://reviews.apache.org/r/61781/#comment259452> It is possible to have a writer write out cache contents between reading the json and hash files. To prevent this, both reads should be guarded by the same acquisition of `__file_lock`, ie. move `with self.__file_lock` to the outermost level, the same way as it is in `persist_cache`. - Attila Doroszlai On Aug. 21, 2017, 6:28 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61781/ > --- > > (Updated Aug. 21, 2017, 6:28 p.m.) > > > Review request for Ambari, Attila Doroszlai, Dmytro Sen, Myroslav > Papirkovskyy, and Sid Wagle. > > > Bugs: AMBARI-21763 > https://issues.apache.org/jira/browse/AMBARI-21763 > > > Repository: ambari > > > Description > --- > > . > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/AlertStatusReporter.py 20cb717 > ambari-agent/src/main/python/ambari_agent/ClusterAlertDefinitionsCache.py > a1f7199 > ambari-agent/src/main/python/ambari_agent/ClusterCache.py 2316866 > ambari-agent/src/main/python/ambari_agent/ClusterTopologyCache.py 559a956 > ambari-agent/src/main/python/ambari_agent/CommandStatusDict.py e7b7e49 > ambari-agent/src/main/python/ambari_agent/ComponentStatusExecutor.py > 66df15a > ambari-agent/src/main/python/ambari_agent/HostStatusReporter.py c60ea36 > ambari-agent/src/main/python/ambari_agent/InitializerModule.py 8208b32 > > ambari-agent/src/main/python/ambari_agent/listeners/AlertDefinitionsEventListener.py > cf72a4d > > ambari-agent/src/main/python/ambari_agent/listeners/ConfigurationEventListener.py > e32c503 > > ambari-agent/src/main/python/ambari_agent/listeners/HostLevelParamsEventListener.py > aee2992 > > ambari-agent/src/main/python/ambari_agent/listeners/MetadataEventListener.py > 1e9b6e7 > > ambari-agent/src/main/python/ambari_agent/listeners/TopologyEventListener.py > 19a1d32 > ambari-agent/src/main/python/ambari_agent/security.py a505658 > ambari-agent/src/test/python/ambari_agent/TestAgentStompResponses.py > 55c489f > > > Diff: https://reviews.apache.org/r/61781/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 61786: AMBARI-21768. Spark History Server uses wrong log dir
> On Aug. 21, 2017, 10:59 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > > Lines 380-381 (patched) > > <https://reviews.apache.org/r/61786/diff/1/?file=1800806#file1800806line380> > > > > I thought that we went through properties like this and changed them to > > "hdp" ... Wouldn't this cause a problem with that replacement? "Change stack references" happens before "Upgrade service configs", so the "iop" introduced here is not subject to replacement. - Attila --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61786/#review183388 ------- On Aug. 21, 2017, 6:23 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61786/ > --- > > (Updated Aug. 21, 2017, 6:23 p.m.) > > > Review request for Ambari, Bikas Saha, Di Li, Jonathan Hurley, Sumit Mohanty, > Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21768 > https://issues.apache.org/jira/browse/AMBARI-21768 > > > Repository: ambari > > > Description > --- > > * Get rid of unnecessary `-Dspark.history.fs.logDirectory`. The value is > taken from `spark-defaults` anyway. > * Tweak `spark.eventLog.dir` and `spark.history.fs.logDirectory` so that they > point to the cluster's existing Spark log directory. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > ad20bf9d9427b6891c4d5944fa3b3e347dbffe12 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > b8c23bbb5292d374d4fb53b9b43a97ebcb1853cd > > > Diff: https://reviews.apache.org/r/61786/diff/1/ > > > Testing > --- > > Manual test according to steps in the bug. > Tested with both default and customized log directory location. > Verified that both pre-upgrade and post-upgrade jobs are shown in Spark > History Server UI. > > > Thanks, > > Attila Doroszlai > >
