We prototyped Mistral / TaskFlow integration and have a follow-up discussions.
SUMMARY: Mistral (Workflow Service) can embed TaskFlow as a workflow library,
with some required modifications to function resliently as a service, and for
smooth integration. However, the TaskFlow flow controls are insufficient for
Mistral use cases.
Details discussed on other thirds.
The prototype scope - [0]; code and discussion - [1] and techical highlights -
[2].
DETAILS:
1) Embedding TaskFlow inside Mistral:
* Required: make the engine "lazy" [3], [4].This is required to support
long-running delegates and not loose tasks when engine manager process restarts.
* Persistence: need clarity how to replace or mix-in TaskFlow persistence with
Mistral persistence. Renat is taking a look.
* Declaring Flows in YAML DSL: done for simplest flow. Need to prototype for
data flow. Rich flow controls are missing in TaskFlow for a representative
prototype.
* ActionRunners vs Taskflow Workers - not prototyped. Not a risk: both Mistral
and TaskFlow implementations work. But we shall resolve the overlap.
* Ignored for now - unlikely any risks: Keystone integration, Mistral event
scheduler, Mistral declarative services and action definition.
2) TaskFlow library features
* Must: flow control - conditional transitions, references, expression
evaluation, to express real-life workflows [5]. The required flow control
primitives are 1) repeater 2) flow in flow 3) direct transition 4) conditional
transition 5) multiple data. TaskFlow has 1) and 2), need to add 3/4/5.
* Other details and smaller requests are in the discussion [1]
3) Next Steps proposed:
* Mistal team: summarize the requirements discussed and agreed on [2] and [3]
* Mistral team: code sample (tests?) on how Mistral would like to consume
TaskFlow lazy engine
* Taskflow team: Provide a design for alternative TaskExecutor approach
(prototypes, boxes, arrows, crayons :))
* Decide on lazy engine
* Move the discussion on other elements on integration.
References:
[0] The scope of the prototype:
https://etherpad.openstack.org/p/mistral-taskflow-prototype
[1] Prototype code and discussion https://github.com/enykeev/mistral/pull/1
[2] Techical summary
http://lists.openstack.org/pipermail/openstack-dev/2014-April/032461.html
[3] Email discussion on TaskFlow lazy eninge
http://lists.openstack.org/pipermail/openstack-dev/2014-March/031134.html
[4] IRC discussion Mistral/Taskflow http://paste.openstack.org/show/75389/
[5] Use cases https://github.com/dzimine/mistral-workflows/tree/add-usecases
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev