This is not supposed to be from 'Outlook', haha.

Using my adjusted mail account so that its don't go to spam on the receiving 
end due to DMARC/DKIM issues...

My fault ;)


On Jul 10, 2014, at 12:51 PM, Outlook <> wrote:

> On Jul 10, 2014, at 3:48 AM, Yuriy Taraday <> wrote:
>> On Wed, Jul 9, 2014 at 7:39 PM, Clint Byrum <> wrote:
>> Excerpts from Yuriy Taraday's message of 2014-07-09 03:36:00 -0700:
>> > On Tue, Jul 8, 2014 at 11:31 PM, Joshua Harlow <>
>> > wrote:
>> >
>> > > I think clints response was likely better than what I can write here, but
>> > > I'll add-on a few things,
>> > >
>> > >
>> > > >How do you write such code using taskflow?
>> > > >
>> > > >  @asyncio.coroutine
>> > > >  def foo(self):
>> > > >      result = yield from some_async_op(...)
>> > > >      return do_stuff(result)
>> > >
>> > > The idea (at a very high level) is that users don't write this;
>> > >
>> > > What users do write is a workflow, maybe the following (pseudocode):
>> > >
>> > > # Define the pieces of your workflow.
>> > >
>> > > TaskA():
>> > >   def execute():
>> > >       # Do whatever some_async_op did here.
>> > >
>> > >   def revert():
>> > >       # If execute had any side-effects undo them here.
>> > >
>> > > TaskFoo():
>> > >    ...
>> > >
>> > > # Compose them together
>> > >
>> > > flow = linear_flow.Flow("my-stuff").add(TaskA("my-task-a"),
>> > > TaskFoo("my-foo"))
>> > >
>> >
>> > I wouldn't consider this composition very user-friendly.
>> >
> So just to make this understandable, the above is a declarative structure of 
> the work to be done. I'm pretty sure it's general agreed[1] in the 
> programming world that when declarative structures can be used they should be 
> (imho openstack should also follow the same pattern more than it currently 
> does). The above is a declaration of the work to be done and the ordering 
> constraints that must be followed. Its just one of X ways to do this (feel 
> free to contribute other variations of these 'patterns' @ 
> [1] (and many many others).
>> I find it extremely user friendly when I consider that it gives you
>> clear lines of delineation between "the way it should work" and "what
>> to do when it breaks."
>> So does plain Python. But for plain Python you don't have to explicitly use 
>> graph terminology to describe the process.
> I'm not sure where in the above you saw graph terminology. All I see there is 
> a declaration of a pattern that explicitly says run things 1 after the other 
> (linearly).
>> > > # Submit the workflow to an engine, let the engine do the work to execute
>> > > it (and transfer any state between tasks as needed).
>> > >
>> > > The idea here is that when things like this are declaratively specified
>> > > the only thing that matters is that the engine respects that declaration;
>> > > not whether it uses asyncio, eventlet, pigeons, threads, remote
>> > > workers[1]. It also adds some things that are not (imho) possible with
>> > > co-routines (in part since they are at such a low level) like stopping 
>> > > the
>> > > engine after 'my-task-a' runs and shutting off the software, upgrading 
>> > > it,
>> > > restarting it and then picking back up at 'my-foo'.
>> > >
>> >
>> > It's absolutely possible with coroutines and might provide even clearer
>> > view of what's going on. Like this:
>> >
>> > @asyncio.coroutine
>> > def my_workflow(ctx, ...):
>> >     project = yield from ctx.run_task(create_project())
>> >     # Hey, we don't want to be linear. How about parallel tasks?
>> >     volume, network = yield from asyncio.gather(
>> >         ctx.run_task(create_volume(project)),
>> >         ctx.run_task(create_network(project)),
>> >     )
>> >     # We can put anything here - why not branch a bit?
>> >     if create_one_vm:
>> >         yield from ctx.run_task(create_vm(project, network))
>> >     else:
>> >         # Or even loops - why not?
>> >         for i in range(network.num_ips()):
>> >             yield from ctx.run_task(create_vm(project, network))
>> >
>> Sorry but the code above is nothing like the code that Josh shared. When
>> create_network(project) fails, how do we revert its side effects? If we
>> want to resume this flow after reboot, how does that work?
>> I understand that there is a desire to write everything in beautiful
>> python yields, try's, finally's, and excepts. But the reality is that
>> python's stack is lost the moment the process segfaults, power goes out
>> on that PDU, or the admin rolls out a new kernel.
>> We're not saying "asyncio vs. taskflow". I've seen that mistake twice
>> already in this thread. Josh and I are suggesting that if there is a
>> movement to think about coroutines, there should also be some time spent
>> thinking at a high level: "how do we resume tasks, revert side effects,
>> and control flow?"
>> If we embed taskflow deep in the code, we get those things, and we can
>> treat tasks as coroutines and let taskflow's event loop be asyncio just
>> the same. If we embed asyncio deep into the code, we don't get any of
>> the high level functions and we get just as much code churn.
> +1 the declaration of what is to do is not connected to the how to do it. 
> IMHO most projects (maybe outside of the underlying core ones, this could 
> include oslo,messaging, since it's a RPC layer at the lowest level) don't 
> care about anything but the workflow that they want to have execute 
> successfully or not. To me that implies that those projects only care about 
> said declaration (this seems to be a reasonable assumption as pretty much all 
> of openstack apis are just workflows around a driver api that abstracts away 
> vendors/open-source solutions).
>> > There's no limit to coroutine usage. The only problem is the library that
>> > would bind everything together.
>> > In my example run_task will have to be really smart, keeping track of all
>> > started tasks, results of all finished ones, skipping all tasks that have
>> > already been done (and substituting already generated results).
>> > But all of this is doable. And I find this way of declaring workflows way
>> > more understandable than whatever would it look like with Flow.add's
>> >
>> The way the flow is declared is important, as it leads to more isolated
>> code. The single place where the flow is declared in Josh's example means
>> that the flow can be imported, the state deserialized and inspected,
>> and resumed by any piece of code: an API call, a daemon start up, an
>> admin command, etc.
>> I may be wrong, but it appears to me that the context that you built in
>> your code example is hard, maybe impossible, to resume after a process
>> restart unless _every_ task is entirely idempotent and thus can just be
>> repeated over and over.
>> I must have not stressed this enough in the last paragraph. The point is to 
>> make run_task method very smart. It should do smth like this (yes, I'm 
>> better in Python than English):
>> @asyncio.coroutine
>> def run_task(self, task):
>>     task_id = yield from self.register_task(task)
>>     res = yield from self.get_stored_result(task_id)
>>     if res is not None:
>>         return res
>>     try:
>>         res = yield from task
>>     except Exception as exc:
>>         yield from self.store_error(task_id, exc)
>>         raise exc
>>     else:
>>         yield from self.store_result(task_id, res)
>>         return res
> I think you just created 
>  ;-)
> Except nothing in the above code says about about 'yield from', or other, it 
> just is a run loop that schedules things (currently using 
> concurrent.futures), waits for them to complete  and then does things with 
> there results (aka the completion part). I even had a note about the 1 place 
> (potentially more) that asyncio would plug-in: 
>  (since asyncio has a concurrent.futures equivalent, in fact exactly the 
> same, which imho is a peculiarity of the python stdlib right now, aka the 
> nearly identical code bases that are asyncio.futures and concurrent.futures).
>> So it won't run any task more then once (unless smth very bad happens 
>> between task run and DB update, but noone is safe from that) and it will 
>> remember all errors that happened before.
>> On the outer level all tasks will get stored in the context and if some 
>> error occurs, they will just get reverted e.g. by calling their revert() 
>> method in reverse loop.
>> how do we resume tasks, revert side effects, and control flow?
>> This would allow to resume tasks, revert side effects on error and provide 
>> way better way of describing control flow (plain Python instead new .add() 
>> language).
>> Declaration of control flow is also just a Python method, nothing else. So 
>> you can import it anywhere, start the same task on one node, stop it and 
>> continue running it on another one.
>> I'm not suggesting that taskflow is useless and asyncio is better (apple vs 
>> oranges). I'm saying that using coroutines (asyncio) can improve ways we can 
>> use taskflow and provide clearer method of developing these flows.
>> This was mostly response to the "this is impossible with coroutines". I say 
>> it is possible and it can even be better.
>> -- 
> Agreed, to me it's a comparison of an engine (asyncio) and a car (taskflow), 
> the car has a fuel-tank (the storage layer), a body & wheels (the workflow 
> structure), a transmission (the runner from above) and also an engine 
> (asyncio or other...). So the comparison to me isn't a comparison about 
> better or worse since you can't compare an engine to a car. I totally expect 
> the engine/s to be able to be asyncio based (as pointed out in clints 
> response); even if it's right now concurrent.futures based; I think we can 
> have more of a detailed specification here on how that would work but 
> hopefully the idea is more clear now.
> The following might help visualize this (for those that are more visual 
> types):
> Anyways other peculiarities that I believe coroutines would have, other 
> thoughts would be interesting to hear...
> 1. How would a coroutine structure handle resumption? When the call-graph is 
> the structure (aka in taskflow terminology a 'pattern') and when a coroutine 
> does a 'yield from' in the middle of its execution (to say start another 
> section of the workflow) and then the machine dies that was running this 
> work, upon restart how does the running coroutine (run_task for example could 
> be the thing executing all this) get back into the middle of that coroutine? 
> Since a coroutine can yield multiple exit points (that coroutine can also 
> store local state in variables and other objects), how would the run_task get 
> back into one of those with the prior state? This seems problematic if the 
> workflow structure is the coroutine call-graph structure. If the solution is 
> to have coroutines chained together and no multiple yield points are allowed 
> (or only 1 yield point is allowed) then it seems you just recreated the 
> existing taskflow pattern ;)
>  a) This is imho even a bigger problem when you start running those 
> coroutines on remote machines, since taskflow has awareness of the declared 
> workflow (not necessarily implemented by a graph, which is why the core.png 
> has a 'cloud' around the compilation) it can request that remote workers[2] 
> execute the components of that workflow (and receive there results and 
> repeat...) in parallel. This is one thing that I don't understand how the 
> call-graph as the structure would actually achieve (if the declared workflow 
> is embedded in the call graph there is no visibility into it to actually do 
> this). This feature is key to scale imho (and is pretty commonly done in 
> other parallel frameworks, via message passing, or under other names, note 
> that [3, 4] and other libraries do similar things).
> 2. Introspection, I hope this one is more obvious. When the coroutine 
> call-graph is the workflow there is no easy way to examine it before it 
> executes (and change parts of it for example before it executes). This is a 
> nice feature imho when it's declaratively and explicitly defined, you get the 
> ability to do this. This part is key to handling upgrades that typically 
> happen (for example the a workflow with the 5th task was upgraded to a newer 
> version, we need to stop the service, shut it off, do the code upgrade, 
> restart the service and change 5th task from v1 to v1.1).
> 3. Dataflow: tasks in taskflow can not just declare workflow dependencies but 
> also dataflow dependencies (this is how tasks transfer things from one to 
> another). I suppose the dataflow dependency would mirror to coroutine 
> variables & arguments (except the variables/arguments would need to be 
> persisted somewhere so that it can be passed back in on failure of the 
> service running that coroutine). How is that possible without an abstraction 
> over those variables/arguments (a coroutine can't store these things in local 
> variables since those will be lost)?It would seem like this would need to 
> recreate the persistence & storage layer[5] that taskflow already uses for 
> this purpose to accomplish this.
> [2]
> [3] 
> [4]
> [5]
> Anyway it would be neat to try to make a 'coro' pattern in taskflow and have 
> the existing engine from above work with those types of coroutines seamlessly 
> (I expect this to actually be pretty easy) and let users who want to go about 
> this path try it (as long as they understand the drawbacks this imho brings) 
> then this could be an avenue to enabling both approaches (either way has its 
> own set of drawbacks and advantages). That way the work done for coroutines 
> is at one place and not scattered all over, this then gains all the projects 
> a declarative workflow (where those projects feel this is applicable) that 
> they can use to reliably (and easily scale it) to execute things. 
> -Josh
>> Kind regards, Yuriy.
>> _______________________________________________
>> OpenStack-dev mailing list
> _______________________________________________
> OpenStack-dev mailing list

OpenStack-dev mailing list

Reply via email to