On Sun, Jan 25, 2015 at 2:52 PM, Ben Darnell <[email protected]> wrote:
> On Sun, Jan 25, 2015 at 5:31 PM, Guido van Rossum <[email protected]> > wrote: > >> I'm probably a bit dense today, but without browsing the code I'm not >> entirely sure how your thing works. >> > > Sorry, the implementation in Tornado is in this series of commits: > https://github.com/tornadoweb/tornado/compare/05c3073ce363...841b2d4de32b > OK, not light Sunday afternoon reading. :-) > What type of thing is the single-dispatch dispatching on? >> > > It dispaches on the type of the thing that was yielded. So to handle > Deferred, I have this: > > + @gen.convert_yielded.register(Deferred) > + def _(d): > + f = Future() > + def errback(failure): > + try: > + failure.raiseException() > + # Should never happen, but just in case > + raise Exception("errback called without error") > + except: > + f.set_exc_info(sys.exc_info()) > + d.addCallbacks(f.set_result, errback) > + return f > OK, I think I get this part. I'm so glad I invested a week of my summer vacation two years ago in understanding Deferred <https://groups.google.com/d/msg/python-tulip/ut4vTG-08k8/PWZzUXX9HYIJ>. :-) > I then call tornado.gen.convert_yielded() on the result of g.send() (or > g.throw()) before doing anything else with it. > > >> How would the asyncio and Tornado versions cooperate? >> > > They don't have to cooperate; they can be completely unaware of each > other. I was able to implement this for Tornado without asyncio or Twisted > having anything comparable. > Ah, so we each just register things that handle the other's kind of Future by converting it to one of our own kind, and then the framework of the caller (the generator that calls yield) gets to keep control. Right? I'm a little worried (again not yet having fully internalized the whole process) that there might be some cases where frameworks keep passing the ball without making progress, but I can also easily imagine that that's not how it works. > > >> >> Perhaps you can submit a patch for asyncio to clarify the proposal? >> > > Sure. Is the tulip project on code.google.com still the place for that or > is new development happening in the cpython repo? > It doesn't really matter -- we merge back and forth these days. You may find the asyncio repo easier to deal with. > > -Ben > > >> >> On Sun, Jan 25, 2015 at 9:42 AM, Ben Darnell <[email protected]> wrote: >> >>> In Tornado 4.1 [1], I've added a functools.singledispatch-based registry >>> for objects that can be yielded in coroutines. This allows for Future-like >>> objects from different frameworks to be mixed seamlessly in the same >>> coroutine (We currently have support for tornado.concurrent.Future, >>> concurrent.futures.Future, asyncio.Future[2], and >>> twisted.internet.defer.Deferred) >>> >>> I'd like to suggest that asyncio add something similar to facilitate the >>> use of libraries from different frameworks in the same coroutine, not just >>> sharing the same event loop. It would be a pretty simple change: just call >>> a function decorated with @singledispatch (whose default implementation is >>> the identity function) before handling the result in Task._step. >>> >>> I've included a simple example of a hybrid tornado/asyncio coroutine >>> below. >>> >>> -Ben >>> >>> [1] Currently in beta, installable with `pip install >>> https://github.com/tornadoweb/tornado/archive/v4.1.0b1.zip` >>> <https://github.com/tornadoweb/tornado/archive/v4.1.0b1.zip> >>> [2] The future objects are all similar enough that they don't really >>> need this level of abstraction, but they serve as a proof of concept. >>> >>> # requirements: >>> # python3.4 (for 3.3, add asyncio and singledispatch) >>> # aiohttp >>> # https://github.com/tornadoweb/tornado/archive/v4.1.0b1.zip >>> import aiohttp >>> import asyncio >>> import tornado.gen >>> import tornado.httpclient >>> import tornado.ioloop >>> import tornado.platform.asyncio >>> >>> @tornado.gen.coroutine >>> def main(): >>> t_client = tornado.httpclient.AsyncHTTPClient() >>> t_response = yield t_client.fetch('http://www.google.com') >>> print('tornado: read %d bytes with status code %d' % >>> (len(t_response.body), t_response.code)) >>> >>> a_response = yield from aiohttp.request('GET', ' >>> http://www.google.com') >>> a_body = yield from a_response.read() >>> print('aiohttp: read %d bytes with status code %d' % >>> (len(a_body), a_response.status)) >>> >>> # Alternate python2-compatible syntax >>> a_response2 = yield asyncio.async(aiohttp.request('GET', ' >>> http://www.google.com')) >>> a_body2 = yield asyncio.async(a_response2.read()) >>> print('aiohttp2: read %d bytes with status code %d' % >>> (len(a_body2), a_response2.status)) >>> >>> if __name__ == '__main__': >>> >>> tornado.ioloop.IOLoop.configure(tornado.platform.asyncio.AsyncIOMainLoop) >>> tornado.ioloop.IOLoop.current().run_sync(main) >>> >>> >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> > > -- --Guido van Rossum (python.org/~guido)
