A follow up kind of question, Why not just create a eventlet "micro" version that underneath uses asyncio? Then code isn't riddled with yield from or Return() or similar. It seems possible (maybe?) to replace the eventlet hub with one that uses asyncio? This is then a nice change for all those who are using eventlet.
Btw great work with trollis! Sent from my really tiny device... > On Feb 4, 2014, at 5:45 AM, "victor stinner" <victor.stin...@enovance.com> > wrote: > > Hi, > > I would like to replace eventlet with asyncio in OpenStack for the > asynchronous programming. The new asyncio module has a better design and is > less "magical". It is now part of python 3.4 arguably becoming the de-facto > standard for asynchronous programming in Python world. > > > The asyncio module is a new module of Python 3.4 written by Guido van Rossum > as an abstraction of existing event loop (Twisted, Tornado, eventlet, etc.). > In fact, it's more than an abstraction: it has its own event loop and can be > used alone. For the background, read the PEP: > > http://www.python.org/dev/peps/pep-3156/ > "Asynchronous IO Support Rebooted: the "asyncio" Module" > > For more information on asyncio, see its documentation: > > http://docs.python.org/dev/library/asyncio.html > > The asyncio module is also known as "Tulip" which is third-party project > written for Python 3.3. Tulip was written first and then it was integrated in > Python 3.4. > > http://code.google.com/p/tulip/ > > The main difference between eventlet and asyncio is that context switching > between two concurrent tasks is explicit. When a task "blocks" (ex: wait for > an event), it should use the "yield from" syntax which will switch to the > next task: it's similar to "greenlet.switch" in eventlet. So it becomes > obvious which parts of the code do switch and which don't. With eventlet, you > have to read the source code of a function before calling it to check if it > may call greenlet.switch() or not, and so debugging is more difficult. Or > worse, the function may switch in a new version of a module, you won't notice > the change. > > The asyncio module handles various kind of events: sockets, pipes, > subprocesses, UNIX signals, etc. All these things are handled in a single > event loop. You can uses an "exector" to run a blocking task in a pool of > thread. There is a "low-level" API using transports and protocols, similar to > Twisted transports and protocols. But there is also a "high-level" API using > streams, which gives a syntax close to eventlet (except that you have to add > "yield from"). See an example to send an HTTP request and print received HTTP > headers, the "yield from reader.readline()" instruction "blocks" until it > gets a full line: > > http://docs.python.org/dev/library/asyncio-stream.html#example > > > The problem is that the asyncio module was written for Python 3.3, whereas > OpenStack is not fully Python 3 compatible (yet). To easy the transition I > have ported asyncio on Python 2, it's the new Trollis project which supports > Python 2.6-3.4: > > https://bitbucket.org/enovance/trollius > > The Trollius API is the same than asyncio, the main difference is the syntax > in coroutines: "yield from task" must be written "yield task", and "return > value" must be written "raise Return(value)". > > > > The first step to move from eventlet to asyncio is a new executor using > Trollius in Olso Messaging: > > https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio > https://review.openstack.org/#/c/70948 > > > The transition from eventlet to asyncio can be done step by step. Thanks to > the greenio project, asyncio can reuse the greenlet event loop (and so run in > the main thread). So asyncio and eventlet become "compatible". While asyncio > can also run its own event loop in a separated thread. > > If eventlet is completely replaced with asyncio in a project, greenio can be > dropped, and asyncio event loop can be run its own event loop in the main > thread. > > When OpenStack will be compatible with Python 3.3, it will be possible to use > the builtin asyncio module of Python 3.4 directly instead of Trollus. Since > "yield from" is incompatible with Python 2, some parts of the code may need > to have two versions (one for Python 2, one for Python 3) if we want to use > the "Python 3 flavor" of asyncio... or Python 2 support might be simplify > dropped. > > Victor > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
