-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34370/
-----------------------------------------------------------
Review request for Ambari and Robert Nettleton.
Bugs: AMBARI-11226
https://issues.apache.org/jira/browse/AMBARI-11226
Repository: ambari
Description
-------
Steps to reproduce:
1. Setup a 3-node cluster (using vagrant or other option)
2. Using the UI, startup a cluster that includes: HDFS, Yarn/MR, Tez,
Zookeeper, Ambari Metrics. There is no need to enable HA to reproduce this
problem.
3. When this cluster deploys successfully, export a Blueprint from this cluster
using the REST API:
api/v1/clusters/clusterone?format=blueprint
The resulting Blueprint will include many invalid mappings. I discovered this
problem yesterday while attempting to run a "round-trip" test (start cluster,
export Blueprint, re-create cluster with exported Blueprint). I noticed that
many services, including the NameNode and any services that depend upon it,
were not starting up properly.
I will attach the Blueprint I'm seeing when I export from a running 3-node
cluster with the components listed above.
The relationship between the components in various host groups seems to no
longer match up properly with the exported properties.
Here are a few examples I've found:
1. In the attached Blueprint, the "NAMENODE" component is included in
"host_group_2". The problem is that the exported configuration refers to the
NameNode being in a separate host group:
in "core-site": "fs.defaultFS" : "hdfs://%HOSTGROUP::host_group_1%:8020",
in "hdfs-site: "dfs.namenode.http-address" : "%HOSTGROUP::host_group_1%:50070",
"dfs.namenode.https-address" : "%HOSTGROUP::host_group_1%:50470",
It's important to note that the properties above should refer to the host group
that contains the NameNode, but they actually refer to a host group
"host_group_1" that does not include a "NAMENODE".
2. The "SECONDARY_NAMENODE" component is included in "host_group_3", but the
config refers to a different host group:
in "hdfs-site" : "dfs.namenode.secondary.http-address" :
"%HOSTGROUP::host_group_2%:50090",
Again, the problem is the same: the properties refer to services on a given
host group, but the services are not there.
There are probably more examples of this in the Blueprint, but these are the
most prominent.
This causes any attempt to deploy with this Blueprint to fail, since the
NameNode cannot start with invalid configuration. Subsequently, any services
that are clients of the NameNode fail,since they attempt to connect to a
service that is not deployed on the configured port.
This will cause any attempt to "round-trip" deploy an exported Blueprint to
fail.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java
9318db9
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequestTest.java
PRE-CREATION
Diff: https://reviews.apache.org/r/34370/diff/
Testing
-------
Functional Testing:
- Exported a blueprint from a cluster created via UI and blueprint and
confirmed correct component -> hostgroup mapping
Unit Testing:
- Created new unit test for this issue
- Currently running entire unit test suite and will update review with unit
test output summary
Thanks,
John Speidel
