----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41660/#review111691 -----------------------------------------------------------
Ship it! Ship It! - Sid Wagle On Dec. 22, 2015, 11:48 p.m., Aravindan Vijayan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/41660/ > ----------------------------------------------------------- > > (Updated Dec. 22, 2015, 11:48 p.m.) > > > Review request for Ambari, Dmytro Sen, Jaimin Jetly, Sumit Mohanty, and Sid > Wagle. > > > Bugs: AMBARI-14445 > https://issues.apache.org/jira/browse/AMBARI-14445 > > > Repository: ambari > > > Description > ------- > > This patch involves changing of AMS in distributed mode to use cluster > zookeeper instead of local zookeeper. > > Summary of Changes in different areas. > > AMS service side > 1. Added new config amshbase.zookeeper.quorum in ams-site > 2. Moved the zookeeper.znode.parent config item from ams-hbase-security-site > to ams-hbase-site > 3. Changed the default value of zookeeper.znode.parent in configs and Java > code to /ams-hbase-unsecure to make sure there is no conflict with HBase > service. > 4. Remove -Dzookeeper.sasl.client.username={{zk_servicename}} in ams-env and > ams-hbase-env content. > > Stack Advisor > 1. Recommend cluster zookeeper quorum to amshbase.zookeeper.quorum in > ams-site. > 2. Recommend and validate correct ports for > hbase.zookeeper.property.clientPort (cluster zookeeper port for distributed > and default 61181 for embedded) > 3. Minor refactoring of util method in stack_advisor to make sure we have 1 > method each for getZookeeperHost and getZookeeperHostsWithPort. > > AMS start up scripts > 1. Do not start hbase zookeeper in distributed mode. > 2. hbase_zookeeper_quorum should resolve to cluster zookeeper quorum and > localhost in distributed and embedded modes respectively. > 3. If embedded mode, delete core-site.xml from collector and ams-hbase conf > directories to make sure embedded mode works in secure cluster. > > UI > 1. Add amshbase.zookeeper.quorum in ams-site to the list of "watchers" in the > config initialization to make sure cluster zookeeper quorum is captured in > the config. > 2. Add the same config in the UI warning messages when a Zookeeper server is > being added / deleted. > > Upgrade > 1. Move zookeeper.znode.parent from ams-hbase-security-site to ams-hbase-site. > 2. Add amshbase.zookeeper.quorum in ams-site. > 3. Update ams-env and ams-hbase-env to remove > -Dzookeeper.sasl.client.username={{zk_servicename}} > 4. Fix minor bug in UpgradeCatalog220. > > Security > 1. Change in kerberos.json to move zookeeper.znode.parent config item from > ams-hbase-security-site to ams-hbase-site. > 2. Similar changes in ranger upgrade JSON files. > > Others > 1. Remove AMS alert that checks zookeeper state. > > > Diffs > ----- > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultPhoenixDataSource.java > 562049b > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java > e1d256d > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java > 85b142d > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog221.java > a27a2b2 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/alerts.json > 4015590 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml > 96e2bb3 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml > a061006 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml > 5e7bc518 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml > e97cfee > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json > dac60f3 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py > 0726802 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py > 556380e > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py > 87fd476 > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > a723f75 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog220Test.java > 896011a > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog221Test.java > 7cf386e > > ambari-server/src/test/python/stacks/2.0.6/AMBARI_METRICS/test_metrics_collector.py > ab4d006 > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py > 1ee3e7a > ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json > 61a7cdc > > ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json > bba7542 > > ambari-server/src/test/resources/kerberos/test_kerberos_descriptor_2_1_3.json > 3b4dff4 > ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json ca9ac3c > > Diff: https://reviews.apache.org/r/41660/diff/ > > > Testing > ------- > > Manual Testing on embedded & distributed mode in secure & unsecure > environments. > > Ambari server and agent python unit tests pass. > > Java Unit tests passed before last set of changes. Currently being run again. > > Patch submitted through apache. > > Added unit tests for config changes to UpgradeCatalog221. > > > Thanks, > > Aravindan Vijayan > >
