Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review153414
---


Ship it!




Ship It!

- Jonathan Hurley


On Oct. 18, 2016, 2:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 2:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  a49fc09 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
>  2ccbcda 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
>  d96e7b5 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review153397
---


Ship it!




Ship It!

- Laszlo Puskas


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  a49fc09 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
>  2ccbcda 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
>  d96e7b5 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review153396
---


Ship it!




Ship It!

- Sebastian Toader


On Oct. 18, 2016, 8:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 8:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  a49fc09 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
>  2ccbcda 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
>  d96e7b5 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 17, 2016, 2:36 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java,
> >  lines 3074-3076
> > 
> >
> > Need a default-if-not-null here

COMMAND_EXECUTION_TYPE is ConfigurationProperty and has a default value set to 
CommandExecutionType.STAGE.


> On Oct. 17, 2016, 2:36 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java,
> >  lines 43-46
> > 
> >
> > Use Objects.equals() for simplicity

In case of using Objects.equals you still had to cast and compare two fields 
separately. There's also EqualsBuilder but given that there are only two fields 
overall wouldn't be simplier.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152873
---


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 17, 2016, 2:19 p.m., Robert Levas wrote:
> > Has this been tested using both models while enabling Kerberos?

I've tested BP deployment of a secure cluster and Enable/Disable Kerberos, also 
Upgrade. Since this DEPENDENCY_ORDER execution applies only for requests marked 
with property INITIAL_START - theese are START requests created via Blueprints 
- everything elso should execute in same manner as before.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152841
---


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 17, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java,
> >  lines 874-875
> > 
> >
> > Why check both here? Isn't it enough to simply check the stage's 
> > configure role ordering? 
> > 
> > Also - you can just return false immediately if it's not 
> > DEPENDENCY_ORDERED so you don't have to if-block out the entire logic of 
> > the method.

Normally would be enough to check only on stage, but I want to be able I can 
enable/disable this feature from ambari.properties.


> On Oct. 17, 2016, 9:37 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java,
> >  line 69
> > 
> >
> > This feels very wrong to hard-code into something like a scheduler. It 
> > should be declarative somehow. Same for the other components below...

Earlier this was hard coded in RoleCommandOrder, I've take out from there. 
Ideally this would come from outside of Ambari, but may be we could added 
methods like isNamenodeHA etc to the Cluster object. May be an ideal sultion 
could be created in a separate patch, as I don't want to make big modifications 
on normal STAGE flow.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152962
---


On Oct. 18, 2016, 6:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152962
---




ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 870)


Shouldn't this be false initially? If it's not DEPENENCY_ORDERED, then this 
would always return true



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (lines 873 - 874)


Why check both here? Isn't it enough to simply check the stage's configure 
role ordering? 

Also - you can just return false immediately if it's not DEPENDENCY_ORDERED 
so you don't have to if-block out the entire logic of the method.



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 (line 21)


Documentation.



ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 (line 1829)


Can you make the description for the documentation a little more detailed?



ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 (line 69)


This feels very wrong to hard-code into something like a scheduler. It 
should be declarative somehow. Same for the other components below...


- Jonathan Hurley


On Oct. 16, 2016, 4:33 a.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 16, 2016, 4:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Nate Cole]

> On Oct. 14, 2016, 10:40 a.m., Sebastian Toader wrote:
> > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql, line 359
> > 
> >
> > This could be an 'int' column and the values stored in it mapped to he 
> > CommandExecutionType enum.
> 
> Sandor Magyari wrote:
> Well, I've followed the same solution like in case of other enums like 
> TopologyRequest.provision_action, HostRoleCommand.role_command etc.

You have to use this as strings, unless you add a table for lookup.  If you use 
int, it will store the order value.  If someone comes and adds a new enum in 
the middle of the file, then the int order will result in the incorrect enum 
value.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152677
---


On Oct. 16, 2016, 4:33 a.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 16, 2016, 4:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152841
---


Fix it, then Ship it!




Has this been tested using both models while enabling Kerberos?


ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 516)


If you are fixing anything, also fix the space in "will".



ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 (line 3075)


would it be worth doing a `toUpperCase` on the result from `getProperty` 
(if not null) to ensure the case of the property does not cause a failure?



ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 (line 34)


Missing JavaDoc



ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
 (lines 25 - 26)


These should be `private` and possibly `private final`



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 (line 111)


Why is the length here 45 where the field in VARCHAR(32) in the DDL files?


- Robert Levas


On Oct. 16, 2016, 4:33 a.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 16, 2016, 4:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152833
---


Ship it!




Ship It!

- Sebastian Toader


On Oct. 16, 2016, 10:33 a.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 16, 2016, 10:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  4f2580a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
>  2ccbcda 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
>  d96e7b5 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/
---

(Updated Oct. 16, 2016, 8:33 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo Puskas, 
Nate Cole, Sumit Mohanty, and Sebastian Toader.


Changes
---

Fix default value for commandExecutionType in sql scripts.


Bugs: AMBARI-18553
https://issues.apache.org/jira/browse/AMBARI-18553


Repository: ambari


Description
---

Today, the START commands stored in role_command_orders table are bound to 
multiple stages created based on dependencies between components - a component 
in the second stage can only begin its START after the entire first stage is 
done as opposed to just its dependencies in the first stage. This eventually 
increases the overall blueprint deployment time.
The goal is to be able to configure a direct dependency based execution model 
of commands, for now only for Blueprint based deployment commands.

Implementation:
---
When creating stages we set the commandExecutionType to RoleGraph. In case 
commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
created. commandExecutionType is persisted into Stage object / entity as well, 
so ActionScheduler can decide based on commandExecutionType how to execute the 
stage. In case commandExecutionType is set to DEPENDENCY_ORDERED it will filter 
out commands having dependencies on other commands IN_PROGRESS. By default 
commandExecutionType is STAGE_BASED which works as before, creating one or more 
stages dependening on dependecies.
DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
initiated by Blueprint deployment and if Ambari property 
server.stage.command.execution_type = DEPENDENCY_ORDERED.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 8cbfb1e 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
f03d8ea 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 5d8f279 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 5afaba8 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
 bbdb808 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 eaea913 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
 c9ab6f9 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
 625b168 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
 5ca4d88 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 34331ee 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 bd23e00 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
 4f2580a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewURLResourceProviderTest.java
 e223d0f 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java
 a03f05b 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleGraphTest.java
 8bead43 
  
ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java
 d26816c 
  

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152677
---




ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql (line 359)


This could be an 'int' column and the values stored in it mapped to he 
CommandExecutionType enum.


- Sebastian Toader


On Oct. 13, 2016, 5:50 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 13, 2016, 5:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  ef4fc33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  091c6d9 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql f28cdc9 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  4f2580a 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 11, 2016, 6:37 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java,
> >  line 410
> > 
> >
> > Does this only affect fresh install and not other commands, especially 
> > during Rolling/Express upgrade?

Yes only START commands in fresh Blueprint based install and only if is enabled 
in ambari.properties.


> On Oct. 11, 2016, 6:37 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java,
> >  line 78
> > 
> >
> > MasterHostResolver.java already has a method called isNameNodeHA().
> > Not sure if you want to use it.

Yes I saw, would be good to put these checks somewhere else but may be would 
better to do in a separate patch, I don't want change too much existing 
functionality (I the normal way of execution of commands) with this patch.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152173
---


On Oct. 13, 2016, 3:50 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 13, 2016, 3:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  ef4fc33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  091c6d9 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/
---

(Updated Oct. 13, 2016, 3:50 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo Puskas, 
Nate Cole, Sumit Mohanty, and Sebastian Toader.


Changes
---

Change command filtering in ActionScheduler, add more comments.


Bugs: AMBARI-18553
https://issues.apache.org/jira/browse/AMBARI-18553


Repository: ambari


Description
---

Today, the START commands stored in role_command_orders table are bound to 
multiple stages created based on dependencies between components - a component 
in the second stage can only begin its START after the entire first stage is 
done as opposed to just its dependencies in the first stage. This eventually 
increases the overall blueprint deployment time.
The goal is to be able to configure a direct dependency based execution model 
of commands, for now only for Blueprint based deployment commands.

Implementation:
---
When creating stages we set the commandExecutionType to RoleGraph. In case 
commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
created. commandExecutionType is persisted into Stage object / entity as well, 
so ActionScheduler can decide based on commandExecutionType how to execute the 
stage. In case commandExecutionType is set to DEPENDENCY_ORDERED it will filter 
out commands having dependencies on other commands IN_PROGRESS. By default 
commandExecutionType is STAGE_BASED which works as before, creating one or more 
stages dependening on dependecies.
DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
initiated by Blueprint deployment and if Ambari property 
server.stage.command.execution_type = DEPENDENCY_ORDERED.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 8cbfb1e 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
f03d8ea 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 ef4fc33 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 5afaba8 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
 bbdb808 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 eaea913 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
 c9ab6f9 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
 625b168 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
 5ca4d88 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 091c6d9 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql f28cdc9 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 bd23e00 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
 4f2580a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewURLResourceProviderTest.java
 e223d0f 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java
 a03f05b 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleGraphTest.java
 8bead43 
  
ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java
 d26816c 
  

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152173
---



Overall a really good improvement to optimize the install/start time.


ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 410)


Does this only affect fresh install and not other commands, especially 
during Rolling/Express upgrade?



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 (line 21)


Please add some javadoc for how these differ.
Can we shorten "STAGE_BASED" to "STAGE"?



ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 (line 78)


MasterHostResolver.java already has a method called isNameNodeHA().
Not sure if you want to use it.



ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 (line 99)


I would prefer if these attributes, hasGLUSTERFS, isNameNodeHAEnabled, 
isResourceManagerHAEnabled weren't passed to the initialize method, but rather 
set on the object, which could then be initialized without any other params, in 
case the function signature needs to change in the future.


- Alejandro Fernandez


On Oct. 11, 2016, 2:41 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 11, 2016, 2:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  e380ae4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  7659a23 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152166
---




ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 103)


This is a big patch, please give me some time to go through it.


- Alejandro Fernandez


On Oct. 11, 2016, 2:41 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 11, 2016, 2:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  e380ae4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  7659a23 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  091c6d9 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 1c17e1a 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  b2eb738 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  4f2580a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
>  2ccbcda 
>   
> 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/
---

(Updated Oct. 11, 2016, 2:41 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo Puskas, 
Nate Cole, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-18553
https://issues.apache.org/jira/browse/AMBARI-18553


Repository: ambari


Description
---

Today, the START commands stored in role_command_orders table are bound to 
multiple stages created based on dependencies between components - a component 
in the second stage can only begin its START after the entire first stage is 
done as opposed to just its dependencies in the first stage. This eventually 
increases the overall blueprint deployment time.
The goal is to be able to configure a direct dependency based execution model 
of commands, for now only for Blueprint based deployment commands.

Implementation:
---
When creating stages we set the commandExecutionType to RoleGraph. In case 
commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
created. commandExecutionType is persisted into Stage object / entity as well, 
so ActionScheduler can decide based on commandExecutionType how to execute the 
stage. In case commandExecutionType is set to DEPENDENCY_ORDERED it will filter 
out commands having dependencies on other commands IN_PROGRESS. By default 
commandExecutionType is STAGE_BASED which works as before, creating one or more 
stages dependening on dependecies.
DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
initiated by Blueprint deployment and if Ambari property 
server.stage.command.execution_type = DEPENDENCY_ORDERED.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 e380ae4 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
f03d8ea 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 2e850ef 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 1fc9dbf 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 5afaba8 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
 bbdb808 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 7659a23 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
 c9ab6f9 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
 625b168 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
 5ca4d88 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 091c6d9 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 1c17e1a 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 b2eb738 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
 4f2580a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewURLResourceProviderTest.java
 e223d0f 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java
 a03f05b 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleGraphTest.java
 8bead43 
  
ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java
 d26816c 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java
 2aaa2cf 

Diff: https://reviews.apache.org/r/52691/diff/


Testing
---

UT in progress.
Tested with manual cluster deployments.


Thanks,

Sandor Magyari