+1 lots of respect for zane in doing this :) I'm still very much interested in seeing how we can connect taskflow in to your model.
I think the features that you guys were wanting (remote workers) are showing up and hopefully will be all they can be! It helps (imho) that taskflow doesn't connect itself to one model (threaded, asyncio, yielding) since its model is tasks, dependencies, and the order in which those are defined which is separate from how it runs (via engines). Engines are allowed to and encouraged to use threads underneath, asyncio or greenthreads, or remote workers.... -----Original Message----- From: Clint Byrum <cl...@fewbar.com> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Date: Thursday, February 6, 2014 at 12:46 PM To: openstack-dev <openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio >All due respect to Zane who created the scheduler. We simply could not >do what we do without it (and I think one of the first things I asked >for was parallel create ;). > >IMO it is the single most confusing thing in Heat whenever one has to >deal with it. If we could stick to a threading model instead, I would >much prefer that. > >Excerpts from Joshua Harlow's message of 2014-02-06 10:22:24 -0800: >> Has there been any investigation into heat. >> >> Heat has already used parts of the coroutine approach (for better or >> worse). >> >> An example: >> >>https://github.com/openstack/heat/blob/master/heat/engine/scheduler.py#L2 >>30 >> >> """ >> Decorator for a task that needs to drive a subtask. >> >> This is essentially a replacement for the Python 3-only "yield from" >> keyword (PEP 380), using the "yield" keyword that is supported in >> Python 2. For example:: >> """.... >> >> >> I bet trollius would somewhat easily replace a big piece of that code. >> >> -Josh >> >> -----Original Message----- >> From: victor stinner <victor.stin...@enovance.com> >> Reply-To: "OpenStack Development Mailing List (not for usage questions)" >> <openstack-dev@lists.openstack.org> >> Date: Thursday, February 6, 2014 at 1:55 AM >> To: "OpenStack Development Mailing List (not for usage questions)" >> <openstack-dev@lists.openstack.org> >> Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet >> with asyncio >> >> >Sean Dague wrote: >> >> First, very cool! >> > >> >Thanks. >> > >> >> This is very promising work. It might be really interesting to figure >> >> out if there was a smaller project inside of OpenStack that could be >> >> test ported over to this (even as a stackforge project), and >>something >> >> we could run in the gate. >> > >> >Oslo Messaging is a small project, but it's more a library. For a full >> >daemon, my colleague Mehdi Abaakouk has a proof-on-concept for >>Ceilometer >> >replacing eventlet with asyncio. Mehdi told me that he doesn't like to >> >debug eventlet race conditions :-) >> > >> >> Our experience is the OpenStack CI system catches bugs in libraries >>and >> >> underlying components that no one else catches, and definitely >>getting >> >> something running workloads hard on this might be helpful in maturing >> >> Trollius. Basically coevolve it with a piece of OpenStack to know >>that >> >> it can actually work on OpenStack and be a viable path forward. >> > >> >Replacing eventlet with asyncio is a huge change. I don't want to force >> >users to use it right now, nor to do the change in one huge commit. The >> >change will be done step by step, and when possible, optional. For >> >example, in Olso Messaging, you can choose the executor: eventlet or >> >blocking (and I want to add asyncio). >> > >> >Victor >> > >> >_______________________________________________ >> >OpenStack-dev mailing list >> >OpenStack-dev@lists.openstack.org >> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >_______________________________________________ >OpenStack-dev mailing list >OpenStack-dev@lists.openstack.org >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
