-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29276/
-----------------------------------------------------------
Review request for Ambari, Jaimin Jetly, John Speidel, Robert Levas, and Yusaku
Sako.
Bugs: AMBARI-8771
https://issues.apache.org/jira/browse/AMBARI-8771
Repository: ambari
Description
-------
This patch resolves AMBARI-8771.
In previous releases of Ambari Blueprints, it was not possible to deploy an
HDFS NameNode HA cluster using a Blueprint. The existing HA support in Ambari
in the UI supported adding HA support to a running cluster, but Ambari did not
support
creating HA clusters from the initial deployment of the cluster.
This patch adds support for deploying HDFS NameNode HA clusters via the
following changes:
- Modifies the HDFS stack service scripts to handle the startup and
configuration requirements of an initial HDFS HA install, including
handling the startup of the active and standby nodes
- Implements support for two new configuration properties in hadoop-env
that allow the user to specify the initial states of the namenodes, by
declaring the host name of the active and standby nodes. These
properties are used by the stack scripts in order to run the setup
specific to each node type. If these properties are not set by
the user, the Blueprints processor will choose defaults for the
active and stanbdy nodes in the HDFS cluster.
- Updates the Blueprint configuration code to handle quorum journal URL
properties, which use a slightly different syntax than other Hadoop
configuration properties.
- Updates the Blueprint validation code to support a NameNode HA deployment
that does not include the SECONDARY_NAMENODE. Currently, our stacks
require the secondary namenode, and there is no way to express a
cardinality that is optional based on HA status, so this handling
now occurs in the Blueprint processor.
- Adds/updates unit tests for all Java changes in this patch
- Adds/updates unit tests for all Python stack changes in this patch
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessor.java
9cfb635
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
c4071d4
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
e8dbc59
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py
c9c81bb
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
ee8b418
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessorTest.java
8540d8b
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
abc0b15
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 9c65210
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_zkfc.py c358e9e
ambari-server/src/test/python/stacks/2.0.6/configs/ha_bootstrap_active_node.json
PRE-CREATION
ambari-server/src/test/python/stacks/2.0.6/configs/ha_bootstrap_standby_node.json
PRE-CREATION
Diff: https://reviews.apache.org/r/29276/diff/
Testing
-------
1. Ran the ambari-server unit tests: "Tests run: 2451, Failures: 0, Errors: 0,
Skipped: 13"
2. Manually verified that I can start a 3-node HDFS HA Cluster with a
Blueprint. I used the Ambari UI to verify that the cluster started via the
Blueprint sets up the active and standby nodes. I've also used the "hdfs
haadmin" tool to verify the status of the NameNodes setup during a
Blueprint-based HA install.
3. Several times this week, I also started up a cluster with my changes
applied, and then used the UI Wizard to setup NameNode HA, to make sure the
stack changes don't break the UI support for adding HA to an existing cluster.
Thanks,
Robert Nettleton
