-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45347/
-----------------------------------------------------------

(Updated March 30, 2016, 2:46 p.m.)


Review request for Ambari, Ajit Kumar, Jonathan Hurley, Nate Cole, Sumit 
Mohanty, Sebastian Toader, and Sid Wagle.


Changes
-------

BlueprintEntity.getSettings() & setSettings(). Make SettingFactory static. 
UpgradeCatalog240 merge.


Bugs: AMBARI-15592
    https://issues.apache.org/jira/browse/AMBARI-15592


Repository: ambari


Description
-------

AMBARI-15592: Auto-start services - support blueprint deployment.

** Issue: **
Define a JSON structure to specify settings for auto start in a blueprint and 
use that information to enable or disable auto start for components during 
deployment.

** Changes **
Added a new section in the blueprint called "settings" which is a collection of 
various setting names to collection of properties. For auto start, the 
following setting names are used:
*recovery_settings*: {"recovery_enabled" : "true" } specifies that auto start 
should be enabled for all components. Can also be specified within a 
collection, to support a uniform schema.
*service_settings*: Collection of service names and the recovery values. 
[{"name":"HDFS", "recovery_enabled":"false"},{...}] overrides cluster level 
recovery settings.
*component_settings*: Collection of component names and the corresponding 
recovery values.  [{"name":"HDFS_CLIENT", "recovery_enabled":"true"},{...}]. 
Overrides both service settings and recovery settings.


Diffs (updated)
-----

  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java
 8578d6beca91bf411d0c3dafeaee42d2dd23caea 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingEntity.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
 a5e2fa1c7a2adaadc8bbe9de15b913cd9a7ca456 
  ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java 
11311dba0f1174248d24276a41837d7284e41607 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java
 cca28ca1529d6bccdf7a870dab7c317c1a334b1d 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
 bea036421c64b70b4bceffcbd0d13c26020a7ed1 
  ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/SettingFactory.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 d8b84ec8fc1ddd13fa460d6029a735a16217a7e1 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
4a0479eb3b749f332fda306ecc9b9698cde0ac92 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
e40165d72d5d97be28882bbdf61cae5f5be67c9b 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
e85ea496696603b20b0efd8ffd6fb5cd2e9246ea 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
8bcda317c70cce320fa69127e52a786fdd372eac 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
c7d7cff85e1aa502976db3f7902c3d920a9e5a02 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
4bd7a7a9209d4d102d6d0612450280214f9b32b0 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
063db3aa6f4e3f14563180be2b20664f9351d852 
  ambari-server/src/main/resources/META-INF/persistence.xml 
513035f5baf6eacfcc69507069d311418546a09c 
  ambari-server/src/main/resources/properties.json 
01c15f2b1c3564c37e8203ec70f2d2b812135b13 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintEntityTest.java
 c660d19aee1727fd4734c07c0e5130f5bbe3c3cf 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingEntityTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/SettingFactoryTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/SettingTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 5cd3bdce409ccc388c5c455d6a3820dfa7fbe1ee 

Diff: https://reviews.apache.org/r/45347/diff/


Testing
-------

** 1. mvn clean install **

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [4.429s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.035s]
[INFO] Ambari Web ........................................ SUCCESS [30.123s]
[INFO] Ambari Views ...................................... SUCCESS [1.231s]
[INFO] Ambari Admin View ................................. SUCCESS [7.878s]
[INFO] ambari-metrics .................................... SUCCESS [0.360s]
[INFO] Ambari Metrics Common ............................. SUCCESS [0.497s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [1.041s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.567s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.638s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [1.506s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [6.252s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [2.092s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [0.909s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [1:13.560s]
[INFO] Ambari Server ..................................... SUCCESS [2:28.882s]
[INFO] Ambari Functional Tests ........................... SUCCESS [2.142s]
[INFO] Ambari Agent ...................................... SUCCESS [24.309s]
[INFO] Ambari Client ..................................... SUCCESS [0.047s]
[INFO] Ambari Python Client .............................. SUCCESS [1.069s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.121s]
[INFO] Ambari Shell ...................................... SUCCESS [0.040s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.710s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.158s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5:12.200s
[INFO] Finished at: Tue Mar 29 19:11:37 PDT 2016
[INFO] Final Memory: 256M/1137M
[INFO] ------------------------------------------------------------------------

** 2. mvn test -DskipPythonTests 
-Dtest=BlueprintEntityTest,UpgradeCatalog240Test,BlueprintSettingEntityTest,SettingFactoryTest,SettingTest
 **

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.orm.entities.BlueprintEntityTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in 
org.apache.ambari.server.orm.entities.BlueprintEntityTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.orm.entities.BlueprintSettingEntityTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec - in 
org.apache.ambari.server.orm.entities.BlueprintSettingEntityTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.topology.SettingFactoryTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in 
org.apache.ambari.server.topology.SettingFactoryTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.topology.SettingTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in 
org.apache.ambari.server.topology.SettingTest
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.upgrade.UpgradeCatalog240Test
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.873 sec - in 
org.apache.ambari.server.upgrade.UpgradeCatalog240Test

Results :

Tests run: 17, Failures: 0, Errors: 0, Skipped: 0

[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 4300 licence.
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
[INFO] skipping execute as per configuraion
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37.273s
[INFO] Finished at: Tue Mar 29 19:03:27 PDT 2016
[INFO] Final Memory: 55M/1005M
[INFO] ------------------------------------------------------------------------

** 3. Manual tests **

Deployed a 2-node VM and replaced the ambari-server JAR on the server node and 
made a blueprint based deployment. Also tested upgrade:

[root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
@/vagrant/blueprints1.json http://localhost:8080/api/v1/blueprints/bp1

[root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
@/vagrant/hg1.json http://localhost:8080/api/v1/clusters/c1
{
  "href" : "http://localhost:8080/api/v1/clusters/c1/requests/1";,
  "Requests" : {
    "id" : 1,
    "status" : "Accepted"
  }
}

** Blueprint "settings" **

  "settings" : [
    {
    "recovery_settings" : [{
      "recovery_enabled" : "true"
     }
    ]},
    {
    "service_settings" : [
        {
          "name" : "HDFS",
          "recovery_enabled" : "false"
        },
        {
          "name" : "TEZ",
          "recovery_enabled" : "false"
        }
    ]},
    {
    "component_settings" : [
    {
      "name" : "DATANODE",
      "recovery_enabled" : "true"
    }
    ]
    }
  ]
  
** Database: blueprint_setting table **

ambari=> select * from blueprint_setting;
 blueprint_name |    setting_name    |                                      
setting_data
----------------+--------------------+----------------------------------------------------------------------------------------
 bp1            | component_settings | 
[{"recovery_enabled":"true","name":"DATANODE"}]
 bp1            | recovery_settings  | [{"recovery_enabled":"true"}]
 bp1            | service_settings   | 
[{"recovery_enabled":"false","name":"HDFS"},{"recovery_enabled":"false","name":"TEZ"}]
(3 rows)

** Database: servicecomponentdesiredstate table **

ambari=> select * from servicecomponentdesiredstate;
 id |   component_name    | cluster_id | desired_stack_id | desired_version | 
desired_state |  service_name  | recovery_enabled

----+---------------------+------------+------------------+-----------------+---------------+----------------+-----------------
  3 | TEZ_CLIENT          |          2 |                7 | UNKNOWN         | 
INSTALLED     | TEZ            |                0
  5 | MAPREDUCE2_CLIENT   |          2 |                7 | UNKNOWN         | 
INSTALLED     | MAPREDUCE2     |                1
 12 | HDFS_CLIENT         |          2 |                7 | UNKNOWN         | 
INSTALLED     | HDFS           |                0
 13 | YARN_CLIENT         |          2 |                7 | UNKNOWN         | 
INSTALLED     | YARN           |                1
 15 | ZOOKEEPER_CLIENT    |          2 |                7 | UNKNOWN         | 
INSTALLED     | ZOOKEEPER      |                1
  1 | DATANODE            |          2 |                7 | UNKNOWN         | 
STARTED       | HDFS           |                1
  2 | METRICS_COLLECTOR   |          2 |                7 | UNKNOWN         | 
STARTED       | AMBARI_METRICS |                1
  4 | HISTORYSERVER       |          2 |                7 | UNKNOWN         | 
STARTED       | MAPREDUCE2     |                1
  6 | ZOOKEEPER_SERVER    |          2 |                7 | UNKNOWN         | 
STARTED       | ZOOKEEPER      |                1
  7 | APP_TIMELINE_SERVER |          2 |                7 | UNKNOWN         | 
STARTED       | YARN           |                1
  8 | METRICS_MONITOR     |          2 |                7 | UNKNOWN         | 
STARTED       | AMBARI_METRICS |                1
  9 | NAMENODE            |          2 |                7 | UNKNOWN         | 
STARTED       | HDFS           |                0
 10 | SECONDARY_NAMENODE  |          2 |                7 | UNKNOWN         | 
STARTED       | HDFS           |                0
 11 | RESOURCEMANAGER     |          2 |                7 | UNKNOWN         | 
STARTED       | YARN           |                1
 14 | METRICS_GRAFANA     |          2 |                7 | UNKNOWN         | 
STARTED       | AMBARI_METRICS |                1
 16 | NODEMANAGER         |          2 |                7 | UNKNOWN         | 
STARTED       | YARN           |                1
(16 rows)


Thanks,

Nahappan Somasundaram

Reply via email to