Re: [openstack-dev] [Mistral] Global Context and Execution Environment

2014-12-15 Thread Renat Akhmerov
Hi,

 It looked good and I began to write down the summary: 
 https://etherpad.openstack.org/p/mistral-global-context 
 https://etherpad.openstack.org/p/mistral-global-context
Thanks, I left my comments in there.

 What problems are we trying to solve: 
 1) reduce passing the same parameters over and over from parent to child
 2) “automatically” make a parameter accessible to most actions without typing 
 it all over (like auth token) 

I agree that it’s one of the angle from what we’re looking at the problem. 
However, IMO, it’s wider than just these two points. My perspective is that we 
are, first of all, discussing workflow variables’ scoping (see my previous 
email in this thread). So I would rather focus on that. Let’s list all the 
scopes that would make sense, their semantics and use cases where each of them 
could solve particular usability problems (I’m saying “usability problems” 
because it’s really all about usability only).

The reason I’m trying to discuss all this from this point of view is because I 
think we should try to be more formal on things like that. 

 Can #1 be solved by passing input to subworkflows automatically

No, it can’t. “input” is something that gets validated upon workflow execution 
(which happens now) and can’t be arbitrarily passed all the way through because 
of that. If we introduce something like “global” scope then we can always pass 
variables of this scope down to nested workflows using a separate mechanism 
(i.e. different parameter of start_workflow() method). 

 Can #2 be solved somehow else? Default passing of arbitrary parameters to 
 action seems like breaking abstraction

Yes, unless explicitly specified I would not give actions more than they need. 
Encapsulation has been proven to be a good thing.

 Thoughts? need to brainstorm further….

Just once again, I appeal to talk about scopes, their semantics and use cases 
purely from workflow language (DSL) and API standpoint because I’m afraid 
otherwise we could bury ourselves under a pile of minor technical details. 
Specification first, implementation second.

Thanks

Renat Akhmerov
@ Mirantis Inc.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Global Context and Execution Environment

2014-12-12 Thread W Chan
Renat, Dmitri,

On supplying the global context into the workflow execution...

In addition to Renat's proposal, I have a few here.

1) Pass them implicitly in start_workflow as another kwargs in the
**params.  But on thinking, we should probably make global context
explicitly defined in the WF spec.  Passing them implicitly may be hard to
follow during troubleshooting where the value comes from by looking at the
WF spec.  Plus there will be cases where WF authors want it explicitly
defined. Still debating here...

inputs = {...}
globals = {...}
start_workflow('my_workflow', inputs, globals=globals)

2) Instead of adding to the WF spec, what if we change the scope in
existing input params?  For example, inputs defined in the top workflow by
default is visible to all subflows (pass down to workflow task on
run_workflow) and tasks (passed to action on execution).

3) Add to the WF spec

workflow:
type: direct
global:
- global1
- global2
input:
- input1
- input2

Winson
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Global Context and Execution Environment

2014-12-12 Thread Dmitri Zimine
Winson, Lakshmi, Renat: 

It looked good and I began to write down the summary: 
https://etherpad.openstack.org/p/mistral-global-context

But than realized that it’s not safe to assume from the action, that the global 
context will be supplied as part of API call. 
Check it out in the etherpad.

What problems are we trying to solve: 
1) reduce passing the same parameters over and over from parent to child
2) “automatically” make a parameter accessible to most actions without typing 
it all over (like auth token) 

Can #1 be solved by passing input to subworkflows automatically
Can #2 be solved somehow else? Default passing of arbitrary parameters to 
action seems like breaking abstraction.

Thoughts? need to brainstorm further….

DZ 


On Dec 12, 2014, at 12:54 PM, W Chan m4d.co...@gmail.com wrote:

 Renat, Dmitri,
 
 On supplying the global context into the workflow execution...
 
 In addition to Renat's proposal, I have a few here.
 
 1) Pass them implicitly in start_workflow as another kwargs in the **params.  
 But on thinking, we should probably make global context explicitly defined in 
 the WF spec.  Passing them implicitly may be hard to follow during 
 troubleshooting where the value comes from by looking at the WF spec.  Plus 
 there will be cases where WF authors want it explicitly defined. Still 
 debating here...
 
 inputs = {...}
 globals = {...}
 start_workflow('my_workflow', inputs, globals=globals)
 
 2) Instead of adding to the WF spec, what if we change the scope in existing 
 input params?  For example, inputs defined in the top workflow by default is 
 visible to all subflows (pass down to workflow task on run_workflow) and 
 tasks (passed to action on execution).
 
 3) Add to the WF spec
 
 workflow:
 type: direct
 global:
 - global1
 - global2
 input:
 - input1
 - input2
 
 Winson
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Global Context and Execution Environment

2014-12-10 Thread Renat Akhmerov
Winson, ok, I got the idea.

Just a crazy idea that came to my mind. What if we just mark some of the input 
parameters as “global”? For example,

wf:
  type: direct
  input:
- param1
- param2: global

One way or another we’re talking about different scopes. I see the following 
possible scopes:

* local - default scope, only current workflow tasks can see it
* global - all entities can see it: this workflow itself (its tasks), its 
nested workflows and actions
* workflow - only this workflow and actions called from this workflow can see it

However, if we follow that path we would need to change how Mistral validates 
workflow input parameters. Currently, if we pass something into workflow it 
must be declared as an input parameter. In case of “global” scope and nested 
workflows this mechanism is too primitive because a nested workflow may get 
something that it doesn’t expect. So it may not be that straightforward.

Thoughts?

Just in case, I’ll repeat related BPs from another thread:

* https://blueprints.launchpad.net/mistral/+spec/mistral-execution-environment 
https://blueprints.launchpad.net/mistral/+spec/mistral-execution-environment
* https://blueprints.launchpad.net/mistral/+spec/mistral-global-context 
https://blueprints.launchpad.net/mistral/+spec/mistral-global-context
* https://blueprints.launchpad.net/mistral/+spec/mistral-workflow-constants 
https://blueprints.launchpad.net/mistral/+spec/mistral-workflow-constants
* https://blueprints.launchpad.net/mistral/+spec/mistral-default-input-values 
https://blueprints.launchpad.net/mistral/+spec/mistral-default-input-values

Renat Akhmerov
@ Mirantis Inc.



 On 10 Dec 2014, at 13:12, W Chan m4d.co...@gmail.com wrote:
 
 Nikolay,
 
 Regarding whether the execution environment BP is the same as this global 
 context BP, I think the difference is in the scope of the variables.  The 
 global context that I'm proposing is provided to the workflow at execution 
 and is only relevant to this execution.  For example, some contextual 
 information about this specific workflow execution (i.e. a reference to a 
 record in an external system related such as a service ticket ID or CMDB 
 record ID).  The values do not necessary carry across multiple executions.  
 But as I understand, the execution environment configuration is a set of 
 reusable configuration that can be shared across multiple workflow 
 executions.  The fact where action parameters are specified explicitly over 
 and over again is a common problem in the DSL.
 
 Winson
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Mistral] Global Context and Execution Environment

2014-12-09 Thread W Chan
Nikolay,

Regarding whether the execution environment BP is the same as this global
context BP, I think the difference is in the scope of the variables.  The
global context that I'm proposing is provided to the workflow at execution
and is only relevant to this execution.  For example, some contextual
information about this specific workflow execution (i.e. a reference to a
record in an external system related such as a service ticket ID or CMDB
record ID).  The values do not necessary carry across multiple executions.
But as I understand, the execution environment configuration is a set of
reusable configuration that can be shared across multiple workflow
executions.  The fact where action parameters are specified explicitly over
and over again is a common problem in the DSL.

Winson
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev