Sure, its not the fully complete lazy_engine, but piece by piece we can get there.
Of course code/contributions are welcome, as such things will benefit more than just mistral, but openstack as a whole :-) -Josh From: Kirill Izotov <enyk...@stackstorm.com<mailto:enyk...@stackstorm.com>> Date: Monday, April 14, 2014 at 9:02 PM To: Joshua Harlow <harlo...@yahoo-inc.com<mailto:harlo...@yahoo-inc.com>> Cc: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [mistral] [taskflow] Mistral TaskFlow integration summary Thank for pointing that out, Joshua. I had a look on [1] and it seems to me that it might actually do the trick to some degree, though I'm afraid this is still not what we are looking for. While Mistral is asynchronous and event-driven, this particular design is not and would still force us to store the engine in memory and therefore limit our means of scalability. The lazy engine (or better controller) I have proposed is asynchronous at its core and would fit the needs for both of us (since it's much easier to make sync from async, rather than backwards). Regarding the retries, while it might work with the current flow design, I doubt it would work with conditional transitions. The attempt to build a repeater by incapsulating the tasks into sub-flow will basically means that every transition they produce will be in that flow and you can't leave it until they are all finished. The whole idea of sub-flows within the scope of direct conditional transitions is a bit unclear to me (and probably us all) at the moment, though I'm trying to rely on them only as a means to lesser the complexity. [1] https://review.openstack.org/#/c/86470 -- Kirill Izotov пятница, 11 апреля 2014 г. в 23:47, Joshua Harlow написал: Thanks for the write-up krill. Also some adjustments, Both points are good, and putting some of this on @ https://etherpad.openstack.org/p/taskflow-mistral-details so that we can have it actively noted (feel free to adjust it). I think ivan is working on some docs/code/… for the lazy engine idea, so hopefully we can get back soon with that. Lets see what comes out of that effort and iterate on that. For (2), our are mostly correct about unconditional execution although [1] does now change this, and there are a few active reviews that are being worked [3] on to fit this mistral use-case better. I believe [2] can help move in this direction, ivans ideas I think will also push it a little farther to. Of course lets work together to make sure they fit the best so that taskflow & mistral & openstack can be the best it can be (pigeons not included). Can we also make sure the small issues are noted somewhere (maybe in the above etherpad??). Thanks! [1] https://wiki.openstack.org/wiki/TaskFlow#Retries [2] https://review.openstack.org/#/c/86470 [3] https://review.openstack.org/#/q/status:open+project:openstack/taskflow,n,z From: Kirill Izotov <enyk...@stackstorm.com<mailto:enyk...@stackstorm.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Thursday, April 10, 2014 at 9:20 PM To: "OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>" <OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>> Subject: [openstack-dev] [mistral] [taskflow] Mistral TaskFlow integration summary Hi everyone, This is a summary to the prototype integration we did not too long ago: http://github.com/enykeev/mistral/pull/1. Hope it would shed some light on the aspects of the integration we are struggling with. There is a possibility to build Mistral on top of TaskFlow as a library, but in order to meet the requirements dictated by Mistral users and use cases, both Mistral and TaskFlow should change. There are two main sides of the story. One is engine. The other is flow control capabilities. 1) THE ENGINE The current TaskFlow implementation of engine doesn't fit Mistral needs because it is synchronous, it blocks the thread, it requires us to store the reference to the particular engine to be able to get its status and suspend the execution and it lacks long-running task compatibility. To fix this problem in a solid and maintainable way, we need to split the engine into its synchronous and asynchronous counterparts. Lazy engine should be async and atomic, it should not have its own state, instead it should rely on some kind of global state (db or in-memory, depending on a type of application). It should have at least two methods: run and task_complete. Run method should calculate the first batch of tasks and schedule them for executing (either put them in queue or spawn the threads). Task_complete should mark a certain task to be completed and then schedule the next batch of tasks that became available due to resolution of this one. The desired use of lazy engine in Mistral is illustrated here: https://wiki.openstack.org/wiki/Mistral/Blueprints/ActionsDesign#Big_Picture. It should support long running tasks and survive engine process restart without loosing the state of the running actions. So it must be passive (lazy) and persistent. On Mistral side we are using Lazy engine by patching async.run directly to the API (or engine queue) and async.task_complete to the worker queue result channel (and the API for long running tasks). We are still sharing the same graph_analyzer, but instead of relying on loop and Futures, we are handling the execution ourselves in a scalable and robust way. Then, on top of it you can build a sync engine by introducing Futures. You are using async.run() to schedule tasks by transforming them to Futures and then starting a loop, checking Futures for completion and sending their results to async.task_complete() which would produce even more Futures to check over. Just the same way TaskFlow do it right now. The reason I'm proposing to extract Futures from async engine is because they won't work if we have multiple engine processes that should handle the task results concurrently (and without that there will be no scalability). 2) THE FLOW CONTROL CAPABILITIES Since we treat TaskFlow as a library we expect them to provide us with a number of primitives to build our workflow with them. Most important of them to us for the moment are Direct Transitions, and Conditional Transitions. The current implementation of flow transitions in TaskFlow are built on top of data flow dependencies where each task provides some data to the flow and requires some data to be present prior being executed. In other words, you are starting to build your flow tree from the last task through the first one by adding their requirements to the tree. All the tasks of successfully finished flow should be successfully finished too. If one of the tasks finishes with error, the whole flow will be reverted back to its initial state unconditionally. At the same time, Mistral use cases require direct control on the order of the task execution, with top-to-bottom scheme where the next task will be determined based on the results of the execution of the current one. This way to successfully finish a flow you don't have to execute all tasks in it. Besides, the error in execution of a particular task may cause execution of another one. The workflow examples (in pseudo DSL) are here: https://github.com/dzimine/mistral-workflows/tree/add-usecases There is also a handful of small issues, but these two differences cover most basic parts of TaskFlow thus block us from integration and require substantial changes in TaskFlow engine design. Inability to make such changes will directly result in Mistral not being able to meet its requirements and thus rendering the whole project useless for its users. -- Kirill Izotov
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
