Thanks for taking this on, Victor. Do you have a small eventlet program example that you can show the transformation to asyncio?
Thanks, -peter On Tue, Feb 4, 2014 at 8:38 AM, victor stinner <[email protected]> wrote: > Hi, > > I would like to replace eventlet with asyncio in OpenStack for the > asynchronous programming. The new asyncio module has a better design and is > less "magical". It is now part of python 3.4 arguably becoming the de-facto > standard for asynchronous programming in Python world. > > > The asyncio module is a new module of Python 3.4 written by Guido van Rossum > as an abstraction of existing event loop (Twisted, Tornado, eventlet, etc.). > In fact, it's more than an abstraction: it has its own event loop and can be > used alone. For the background, read the PEP: > > http://www.python.org/dev/peps/pep-3156/ > "Asynchronous IO Support Rebooted: the "asyncio" Module" > > For more information on asyncio, see its documentation: > > http://docs.python.org/dev/library/asyncio.html > > The asyncio module is also known as "Tulip" which is third-party project > written for Python 3.3. Tulip was written first and then it was integrated in > Python 3.4. > > http://code.google.com/p/tulip/ > > The main difference between eventlet and asyncio is that context switching > between two concurrent tasks is explicit. When a task "blocks" (ex: wait for > an event), it should use the "yield from" syntax which will switch to the > next task: it's similar to "greenlet.switch" in eventlet. So it becomes > obvious which parts of the code do switch and which don't. With eventlet, you > have to read the source code of a function before calling it to check if it > may call greenlet.switch() or not, and so debugging is more difficult. Or > worse, the function may switch in a new version of a module, you won't notice > the change. > > The asyncio module handles various kind of events: sockets, pipes, > subprocesses, UNIX signals, etc. All these things are handled in a single > event loop. You can uses an "exector" to run a blocking task in a pool of > thread. There is a "low-level" API using transports and protocols, similar to > Twisted transports and protocols. But there is also a "high-level" API using > streams, which gives a syntax close to eventlet (except that you have to add > "yield from"). See an example to send an HTTP request and print received HTTP > headers, the "yield from reader.readline()" instruction "blocks" until it > gets a full line: > > http://docs.python.org/dev/library/asyncio-stream.html#example > > > The problem is that the asyncio module was written for Python 3.3, whereas > OpenStack is not fully Python 3 compatible (yet). To easy the transition I > have ported asyncio on Python 2, it's the new Trollis project which supports > Python 2.6-3.4: > > https://bitbucket.org/enovance/trollius > > The Trollius API is the same than asyncio, the main difference is the syntax > in coroutines: "yield from task" must be written "yield task", and "return > value" must be written "raise Return(value)". > > > > The first step to move from eventlet to asyncio is a new executor using > Trollius in Olso Messaging: > > https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio > https://review.openstack.org/#/c/70948 > > > The transition from eventlet to asyncio can be done step by step. Thanks to > the greenio project, asyncio can reuse the greenlet event loop (and so run in > the main thread). So asyncio and eventlet become "compatible". While asyncio > can also run its own event loop in a separated thread. > > If eventlet is completely replaced with asyncio in a project, greenio can be > dropped, and asyncio event loop can be run its own event loop in the main > thread. > > When OpenStack will be compatible with Python 3.3, it will be possible to use > the builtin asyncio module of Python 3.4 directly instead of Trollus. Since > "yield from" is incompatible with Python 2, some parts of the code may need > to have two versions (one for Python 2, one for Python 3) if we want to use > the "Python 3 flavor" of asyncio... or Python 2 support might be simplify > dropped. > > Victor > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
