-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41660/
-----------------------------------------------------------
Review request for Ambari, Dmytro Sen, Jaimin Jetly, Sumit Mohanty, and Sid
Wagle.
Bugs: AMBARI-14445
https://issues.apache.org/jira/browse/AMBARI-14445
Repository: ambari
Description
-------
This patch involves changing of AMS in distributed mode to use cluster
zookeeper instead of local zookeeper.
Summary of Changes in different areas.
AMS service side
1. Added new config amshbase.zookeeper.quorum in ams-site
2. Moved the zookeeper.znode.parent config item from ams-hbase-security-site to
ams-hbase-site
3. Changed the default value of zookeeper.znode.parent in configs and Java code
to /ams-hbase-unsecure to make sure there is no conflict with HBase service.
4. Remove -Dzookeeper.sasl.client.username={{zk_servicename}} in ams-env and
ams-hbase-env content.
Stack Advisor
1. Recommend cluster zookeeper quorum to amshbase.zookeeper.quorum in ams-site.
2. Recommend and validate correct ports for hbase.zookeeper.property.clientPort
(cluster zookeeper port for distributed and default 61181 for embedded)
3. Minor refactoring of util method in stack_advisor to make sure we have 1
method each for getZookeeperHost and getZookeeperHostsWithPort.
AMS start up scripts
1. Do not start hbase zookeeper in distributed mode.
2. hbase_zookeeper_quorum should resolve to cluster zookeeper quorum and
localhost in distributed and embedded modes respectively.
3. If embedded mode, delete core-site.xml from collector and ams-hbase conf
directories to make sure embedded mode works in secure cluster.
UI
1. Add amshbase.zookeeper.quorum in ams-site to the list of "watchers" in the
config initialization to make sure cluster zookeeper quorum is captured in the
config.
2. Add the same config in the UI warning messages when a Zookeeper server is
being added / deleted.
Upgrade
1. Move zookeeper.znode.parent from ams-hbase-security-site to ams-hbase-site.
2. Add amshbase.zookeeper.quorum in ams-site.
3. Update ams-env and ams-hbase-env to remove
-Dzookeeper.sasl.client.username={{zk_servicename}}
4. Fix minor bug in UpgradeCatalog220.
Security
1. Change in kerberos.json to move zookeeper.znode.parent config item from
ams-hbase-security-site to ams-hbase-site.
2. Similar changes in ranger upgrade JSON files.
Others
1. Remove AMS alert that checks zookeeper state.
Diffs
-----
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultPhoenixDataSource.java
562049b
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
e1d256d
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
85b142d
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog221.java
a27a2b2
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/alerts.json
4015590
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
96e2bb3
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
a061006
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml
5e7bc518
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
e97cfee
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
beb0fa9
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
dac60f3
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py
0726802
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py
556380e
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
87fd476
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
a723f75
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog220Test.java
896011a
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog221Test.java
7cf386e
ambari-server/src/test/python/stacks/2.0.6/AMBARI_METRICS/test_metrics_collector.py
ab4d006
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 1ee3e7a
ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json
61a7cdc
ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json
bba7542
ambari-server/src/test/resources/kerberos/test_kerberos_descriptor_2_1_3.json
3b4dff4
ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json ca9ac3c
ambari-web/app/messages.js c0e1e16
ambari-web/app/utils/configs/add_component_config_initializer.js 6fc505c
Diff: https://reviews.apache.org/r/41660/diff/
Testing
-------
Manual Testing on embedded & distributed mode in secure & unsecure environments.
Ambari server and agent python unit tests pass.
Java Unit tests passed before last set of changes. Currently being run again.
Patch submitted through apache.
Added unit tests for config changes to UpgradeCatalog221.
Thanks,
Aravindan Vijayan
