> On March 28, 2016, 2:44 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql, line 424
> > <https://reviews.apache.org/r/45347/diff/2/?file=1315827#file1315827line424>
> >
> >     Let's use an ID here for the PK, so we can reference it in other tables 
> > as a FK.
> >     We can make blueprint_name a UQ
> 
> Alejandro Fernandez wrote:
>     Also, let's give names to the FK and UK.
> 
> Nahappan Somasundaram wrote:
>     Blueprint name per row is not unique. Depending on the numbe of settings, 
> the blueprint_name will be repeated. A sequential ID column can be added for 
> future use as a FK lookup.
> 
> Nahappan Somasundaram wrote:
>     The blueprint_configuration and blueprint_settings tables are read only 
> once during the template deployment. If we want to reference the blueprint 
> name using an UQ id, then the blueprint table needs an ID column. Then 
> blueprint_configuration  and blueprint_settings table need to be modified, 
> which is a lot of ask for one read.
>     
>     In the blueprint_settings table, blueprint_name and setting_name form the 
> composite PK. Adding an ID column in this table does not add much value 
> unless we plan to use it later for quick lookup or search.
> 
> Alejandro Fernandez wrote:
>     Let's use the PK as the ID then. We can enforce uniqueness of name and 
> setting via business logic and/or named UQ.
>     We've ran into problem in the past by not using IDs as PKs.

+1 for id column as PK.


- Nate


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45347/#review125704
-----------------------------------------------------------


On March 27, 2016, 5:51 p.m., Nahappan Somasundaram wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45347/
> -----------------------------------------------------------
> 
> (Updated March 27, 2016, 5:51 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, 
> Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15592
>     https://issues.apache.org/jira/browse/AMBARI-15592
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> AMBARI-15592: Auto-start services - support blueprint deployment.
> 
> ** Issue: **
> Define a JSON structure to specify settings for auto start in a blueprint and 
> use that information to enable or disable auto start for components during 
> deployment.
> 
> ** Changes **
> Added a new section in the blueprint called "settings" which is a collection 
> of various setting names to collection of properties. For auto start, the 
> following setting names are used:
> *recovery_settings*: {"recovery_enabled" : "true" } specifies that auto start 
> should be enabled for all components. Can also be specified within a 
> collection, to support a uniform schema.
> *service_settings*: Collection of service names and the recovery values. 
> [{"name":"HDFS", "recovery_enabled":"false"},{...}] overrides cluster level 
> recovery settings.
> *component_settings*: Collection of component names and the corresponding 
> recovery values.  [{"name":"HDFS_CLIENT", "recovery_enabled":"true"},{...}]. 
> Overrides both service settings and recovery settings.
> 
> 
> Diffs
> -----
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java
>  8578d6beca91bf411d0c3dafeaee42d2dd23caea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntity.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityPK.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
>  87225adc0afafc111b865120fd71742dc4e096f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java 
> 11311dba0f1174248d24276a41837d7284e41607 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java
>  cca28ca1529d6bccdf7a870dab7c317c1a334b1d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
>  bea036421c64b70b4bceffcbd0d13c26020a7ed1 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/Settings.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/SettingsFactory.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  38a3614ae3543ae1de0ea0bfd4191951c6d5c5ff 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
> f8c97ca11040bff414626189591d86d4781be478 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
> b306c0ab7687c6fe05c818acefa2d063e5b111ed 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
> 37744f8bc729e374a1db76f6509a6e85bffb37d0 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
> eba1745d8c7a3e1b677cb8b8fc33d9ae09d0cf68 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> cad0a39d824fdbf03b1ec6cd2a99dec79fa916d2 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> 346af50961fa1b2a41bd0a81121a08ec8ed70a2f 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
> e238a7665dff5a52426be9e334e4a48f53c7dbee 
>   ambari-server/src/main/resources/META-INF/persistence.xml 
> 513035f5baf6eacfcc69507069d311418546a09c 
>   ambari-server/src/main/resources/properties.json 
> 01c15f2b1c3564c37e8203ec70f2d2b812135b13 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintEntityTest.java
>  c660d19aee1727fd4734c07c0e5130f5bbe3c3cf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityPKTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/SettingsFactoryTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/SettingsTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  7b2d797486061a1377622806f7680176fa8ad458 
> 
> Diff: https://reviews.apache.org/r/45347/diff/
> 
> 
> Testing
> -------
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [6.402s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.039s]
> [INFO] Ambari Web ........................................ SUCCESS [24.133s]
> [INFO] Ambari Views ...................................... SUCCESS [1.280s]
> [INFO] Ambari Admin View ................................. SUCCESS [5.725s]
> [INFO] ambari-metrics .................................... SUCCESS [0.346s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [0.459s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [1.077s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.574s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.582s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [1.459s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [6.518s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.164s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.847s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:15.164s]
> [INFO] Ambari Server ..................................... SUCCESS [2:20.006s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [1.010s]
> [INFO] Ambari Agent ...................................... SUCCESS [22.882s]
> [INFO] Ambari Client ..................................... SUCCESS [0.043s]
> [INFO] Ambari Python Client .............................. SUCCESS [0.839s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.054s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.040s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.676s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.182s]
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 4:56.150s
> [INFO] Finished at: Fri Mar 25 17:24:57 PDT 2016
> [INFO] Final Memory: 279M/1202M
> [INFO] 
> ------------------------------------------------------------------------
> 
> ** 2. mvn test -DskipPythonTests 
> -Dtest=BlueprintEntityTest,BlueprintSettingsEntityPKTest,BlueprintSettingsEntityTest,SettingsFactoryTest,SettingsTest
>  **
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.orm.entities.BlueprintEntityTest
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - 
> in org.apache.ambari.server.orm.entities.BlueprintEntityTest
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.orm.entities.BlueprintSettingsEntityPKTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - 
> in org.apache.ambari.server.orm.entities.BlueprintSettingsEntityPKTest
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.orm.entities.BlueprintSettingsEntityTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec - 
> in org.apache.ambari.server.orm.entities.BlueprintSettingsEntityTest
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.topology.SettingsFactoryTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - 
> in org.apache.ambari.server.topology.SettingsFactoryTest
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.topology.SettingsTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec - 
> in org.apache.ambari.server.topology.SettingsTest
> 
> Results :
> 
> Tests run: 16, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
> approved: 4290 licence.
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
> [INFO] skipping execute as per configuraion
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 28.801s
> [INFO] Finished at: Fri Mar 25 17:32:53 PDT 2016
> [INFO] Final Memory: 55M/1014M
> [INFO] 
> ------------------------------------------------------------------------
> 
> 
> ** mvn test -DskipPythonTests -Dtest=UpgradeCatalog240Test **
> 
> 
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.upgrade.UpgradeCatalog240Test
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.649 sec - 
> in org.apache.ambari.server.upgrade.UpgradeCatalog240Test
> 
> Results :
> 
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 33.620s
> [INFO] Finished at: Sun Mar 27 14:49:50 PDT 2016
> [INFO] Final Memory: 55M/1013M
> [INFO] 
> ------------------------------------------------------------------------
> 
> ** 3. Manual tests **
> 
> Deployed a 2-node VM and replaced the ambari-server JAR on the server node 
> and made a blueprint based deployment:
> 
> [root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
> @/vagrant/blueprints1.json http://localhost:8080/api/v1/blueprints/bp1
> 
> [root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
> @/vagrant/hg1.json http://localhost:8080/api/v1/clusters/c1
> {
>   "href" : "http://localhost:8080/api/v1/clusters/c1/requests/1";,
>   "Requests" : {
>     "id" : 1,
>     "status" : "Accepted"
>   }
> }
> 
> ** Blueprint "settings" **
> 
>   "settings" : [
>     {
>     "recovery_settings" : [{
>       "recovery_enabled" : "true"
>      }
>     ]},
>     {
>     "service_settings" : [
>         {
>         "name" : "HDFS",
>         "recovery_enabled" : "false"
>         },
>         {
>         "name" : "TEZ",
>         "recovery_enabled" : "false"
>         }
>     ]},
>     {
>     "component_settings" : [
>     {
>       "name" : "DATANODE",
>       "recovery_enabled" : "true"
>     }
>     ]
>     }
>   ]
>   
> ** Database: blueprint_settings table **
> 
> ambari=> select * from blueprint_settings;
>  blueprint_name |    setting_name    |                                      
> setting_data
> ----------------+--------------------+----------------------------------------------------------------------------------------
>  bp1            | component_settings | 
> [{"recovery_enabled":"true","name":"DATANODE"}]
>  bp1            | recovery_settings  | [{"recovery_enabled":"true"}]
>  bp1            | service_settings   | 
> [{"recovery_enabled":"false","name":"HDFS"},{"recovery_enabled":"false","name":"TEZ"}]
> (3 rows)
> 
> ** Database: servicecomponentdesiredstate table **
> 
> ambari=> select * from servicecomponentdesiredstate;
>  id |   component_name    | cluster_id | desired_stack_id | desired_version | 
> desired_state |  service_name  | recovery_enabled
> 
> ----+---------------------+------------+------------------+-----------------+---------------+----------------+-----------------
>   3 | TEZ_CLIENT          |          2 |                7 | UNKNOWN         | 
> INSTALLED     | TEZ            |                0
>   5 | MAPREDUCE2_CLIENT   |          2 |                7 | UNKNOWN         | 
> INSTALLED     | MAPREDUCE2     |                1
>  12 | HDFS_CLIENT         |          2 |                7 | UNKNOWN         | 
> INSTALLED     | HDFS           |                0
>  13 | YARN_CLIENT         |          2 |                7 | UNKNOWN         | 
> INSTALLED     | YARN           |                1
>  15 | ZOOKEEPER_CLIENT    |          2 |                7 | UNKNOWN         | 
> INSTALLED     | ZOOKEEPER      |                1
>   1 | DATANODE            |          2 |                7 | UNKNOWN         | 
> STARTED       | HDFS           |                1
>   2 | METRICS_COLLECTOR   |          2 |                7 | UNKNOWN         | 
> STARTED       | AMBARI_METRICS |                1
>   4 | HISTORYSERVER       |          2 |                7 | UNKNOWN         | 
> STARTED       | MAPREDUCE2     |                1
>   6 | ZOOKEEPER_SERVER    |          2 |                7 | UNKNOWN         | 
> STARTED       | ZOOKEEPER      |                1
>   7 | APP_TIMELINE_SERVER |          2 |                7 | UNKNOWN         | 
> STARTED       | YARN           |                1
>   8 | METRICS_MONITOR     |          2 |                7 | UNKNOWN         | 
> STARTED       | AMBARI_METRICS |                1
>   9 | NAMENODE            |          2 |                7 | UNKNOWN         | 
> STARTED       | HDFS           |                0
>  10 | SECONDARY_NAMENODE  |          2 |                7 | UNKNOWN         | 
> STARTED       | HDFS           |                0
>  11 | RESOURCEMANAGER     |          2 |                7 | UNKNOWN         | 
> STARTED       | YARN           |                1
>  14 | METRICS_GRAFANA     |          2 |                7 | UNKNOWN         | 
> STARTED       | AMBARI_METRICS |                1
>  16 | NODEMANAGER         |          2 |                7 | UNKNOWN         | 
> STARTED       | YARN           |                1
> (16 rows)
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Reply via email to