-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45347/
-----------------------------------------------------------
(Updated March 29, 2016, 11:44 a.m.)
Review request for Ambari, Ajit Kumar, Jonathan Hurley, Nate Cole, Sumit
Mohanty, Sebastian Toader, and Sid Wagle.
Changes
-------
1. Updated SQL files to have inline named constraints.
2. Removed BlueprintSettingsEntityPK class since we have ID field as the PK.
3. SettingsFactory is now an injectable singleton.
Bugs: AMBARI-15592
https://issues.apache.org/jira/browse/AMBARI-15592
Repository: ambari
Description
-------
AMBARI-15592: Auto-start services - support blueprint deployment.
** Issue: **
Define a JSON structure to specify settings for auto start in a blueprint and
use that information to enable or disable auto start for components during
deployment.
** Changes **
Added a new section in the blueprint called "settings" which is a collection of
various setting names to collection of properties. For auto start, the
following setting names are used:
*recovery_settings*: {"recovery_enabled" : "true" } specifies that auto start
should be enabled for all components. Can also be specified within a
collection, to support a uniform schema.
*service_settings*: Collection of service names and the recovery values.
[{"name":"HDFS", "recovery_enabled":"false"},{...}] overrides cluster level
recovery settings.
*component_settings*: Collection of component names and the corresponding
recovery values. [{"name":"HDFS_CLIENT", "recovery_enabled":"true"},{...}].
Overrides both service settings and recovery settings.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java
8578d6beca91bf411d0c3dafeaee42d2dd23caea
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntity.java
PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
a5e2fa1c7a2adaadc8bbe9de15b913cd9a7ca456
ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java
11311dba0f1174248d24276a41837d7284e41607
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java
cca28ca1529d6bccdf7a870dab7c317c1a334b1d
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
bea036421c64b70b4bceffcbd0d13c26020a7ed1
ambari-server/src/main/java/org/apache/ambari/server/topology/Settings.java
PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/topology/SettingsFactory.java
PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
38a3614ae3543ae1de0ea0bfd4191951c6d5c5ff
ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql
f8c97ca11040bff414626189591d86d4781be478
ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
b306c0ab7687c6fe05c818acefa2d063e5b111ed
ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql
37744f8bc729e374a1db76f6509a6e85bffb37d0
ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
eba1745d8c7a3e1b677cb8b8fc33d9ae09d0cf68
ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql
cad0a39d824fdbf03b1ec6cd2a99dec79fa916d2
ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
346af50961fa1b2a41bd0a81121a08ec8ed70a2f
ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
e238a7665dff5a52426be9e334e4a48f53c7dbee
ambari-server/src/main/resources/META-INF/persistence.xml
513035f5baf6eacfcc69507069d311418546a09c
ambari-server/src/main/resources/properties.json
01c15f2b1c3564c37e8203ec70f2d2b812135b13
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintEntityTest.java
c660d19aee1727fd4734c07c0e5130f5bbe3c3cf
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityTest.java
PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/topology/SettingsFactoryTest.java
PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/topology/SettingsTest.java
PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
7b2d797486061a1377622806f7680176fa8ad458
Diff: https://reviews.apache.org/r/45347/diff/
Testing
-------
** 1. mvn clean install **
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [6.402s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.039s]
[INFO] Ambari Web ........................................ SUCCESS [24.133s]
[INFO] Ambari Views ...................................... SUCCESS [1.280s]
[INFO] Ambari Admin View ................................. SUCCESS [5.725s]
[INFO] ambari-metrics .................................... SUCCESS [0.346s]
[INFO] Ambari Metrics Common ............................. SUCCESS [0.459s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [1.077s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.574s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.582s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [1.459s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [6.518s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [2.164s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [0.847s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [1:15.164s]
[INFO] Ambari Server ..................................... SUCCESS [2:20.006s]
[INFO] Ambari Functional Tests ........................... SUCCESS [1.010s]
[INFO] Ambari Agent ...................................... SUCCESS [22.882s]
[INFO] Ambari Client ..................................... SUCCESS [0.043s]
[INFO] Ambari Python Client .............................. SUCCESS [0.839s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.054s]
[INFO] Ambari Shell ...................................... SUCCESS [0.040s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.676s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.182s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:56.150s
[INFO] Finished at: Fri Mar 25 17:24:57 PDT 2016
[INFO] Final Memory: 279M/1202M
[INFO] ------------------------------------------------------------------------
** 2. mvn test -DskipPythonTests
-Dtest=BlueprintEntityTest,BlueprintSettingsEntityPKTest,BlueprintSettingsEntityTest,SettingsFactoryTest,SettingsTest
**
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.orm.entities.BlueprintEntityTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in
org.apache.ambari.server.orm.entities.BlueprintEntityTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.orm.entities.BlueprintSettingsEntityPKTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - in
org.apache.ambari.server.orm.entities.BlueprintSettingsEntityPKTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.orm.entities.BlueprintSettingsEntityTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec - in
org.apache.ambari.server.orm.entities.BlueprintSettingsEntityTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.topology.SettingsFactoryTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - in
org.apache.ambari.server.topology.SettingsFactoryTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.topology.SettingsTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec - in
org.apache.ambari.server.topology.SettingsTest
Results :
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0
approved: 4290 licence.
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
[INFO] skipping execute as per configuraion
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28.801s
[INFO] Finished at: Fri Mar 25 17:32:53 PDT 2016
[INFO] Final Memory: 55M/1014M
[INFO] ------------------------------------------------------------------------
** mvn test -DskipPythonTests -Dtest=UpgradeCatalog240Test **
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.upgrade.UpgradeCatalog240Test
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.649 sec - in
org.apache.ambari.server.upgrade.UpgradeCatalog240Test
Results :
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33.620s
[INFO] Finished at: Sun Mar 27 14:49:50 PDT 2016
[INFO] Final Memory: 55M/1013M
[INFO] ------------------------------------------------------------------------
** 3. Manual tests **
Deployed a 2-node VM and replaced the ambari-server JAR on the server node and
made a blueprint based deployment:
[root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d
@/vagrant/blueprints1.json http://localhost:8080/api/v1/blueprints/bp1
[root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d
@/vagrant/hg1.json http://localhost:8080/api/v1/clusters/c1
{
"href" : "http://localhost:8080/api/v1/clusters/c1/requests/1";,
"Requests" : {
"id" : 1,
"status" : "Accepted"
}
}
** Blueprint "settings" **
"settings" : [
{
"recovery_settings" : [{
"recovery_enabled" : "true"
}
]},
{
"service_settings" : [
{
"name" : "HDFS",
"recovery_enabled" : "false"
},
{
"name" : "TEZ",
"recovery_enabled" : "false"
}
]},
{
"component_settings" : [
{
"name" : "DATANODE",
"recovery_enabled" : "true"
}
]
}
]
** Database: blueprint_settings table **
ambari=> select * from blueprint_settings;
blueprint_name | setting_name |
setting_data
----------------+--------------------+----------------------------------------------------------------------------------------
bp1 | component_settings |
[{"recovery_enabled":"true","name":"DATANODE"}]
bp1 | recovery_settings | [{"recovery_enabled":"true"}]
bp1 | service_settings |
[{"recovery_enabled":"false","name":"HDFS"},{"recovery_enabled":"false","name":"TEZ"}]
(3 rows)
** Database: servicecomponentdesiredstate table **
ambari=> select * from servicecomponentdesiredstate;
id | component_name | cluster_id | desired_stack_id | desired_version |
desired_state | service_name | recovery_enabled
----+---------------------+------------+------------------+-----------------+---------------+----------------+-----------------
3 | TEZ_CLIENT | 2 | 7 | UNKNOWN |
INSTALLED | TEZ | 0
5 | MAPREDUCE2_CLIENT | 2 | 7 | UNKNOWN |
INSTALLED | MAPREDUCE2 | 1
12 | HDFS_CLIENT | 2 | 7 | UNKNOWN |
INSTALLED | HDFS | 0
13 | YARN_CLIENT | 2 | 7 | UNKNOWN |
INSTALLED | YARN | 1
15 | ZOOKEEPER_CLIENT | 2 | 7 | UNKNOWN |
INSTALLED | ZOOKEEPER | 1
1 | DATANODE | 2 | 7 | UNKNOWN |
STARTED | HDFS | 1
2 | METRICS_COLLECTOR | 2 | 7 | UNKNOWN |
STARTED | AMBARI_METRICS | 1
4 | HISTORYSERVER | 2 | 7 | UNKNOWN |
STARTED | MAPREDUCE2 | 1
6 | ZOOKEEPER_SERVER | 2 | 7 | UNKNOWN |
STARTED | ZOOKEEPER | 1
7 | APP_TIMELINE_SERVER | 2 | 7 | UNKNOWN |
STARTED | YARN | 1
8 | METRICS_MONITOR | 2 | 7 | UNKNOWN |
STARTED | AMBARI_METRICS | 1
9 | NAMENODE | 2 | 7 | UNKNOWN |
STARTED | HDFS | 0
10 | SECONDARY_NAMENODE | 2 | 7 | UNKNOWN |
STARTED | HDFS | 0
11 | RESOURCEMANAGER | 2 | 7 | UNKNOWN |
STARTED | YARN | 1
14 | METRICS_GRAFANA | 2 | 7 | UNKNOWN |
STARTED | AMBARI_METRICS | 1
16 | NODEMANAGER | 2 | 7 | UNKNOWN |
STARTED | YARN | 1
(16 rows)
Thanks,
Nahappan Somasundaram
