On Mon, Mar 25, 2019 at 4:54 PM Ben Darnell <b...@bendarnell.com> wrote:
> On Mon, Mar 25, 2019 at 7:37 PM Guido van Rossum <gu...@python.org> wrote: > >> Thanks for bringing this up -- I think it will be good to get to the >> bottom of this, before the Jupyter folks accidentally get everyone to use >> an approach that is unsound. Maybe they can be redirected to a better >> strategy, or maybe they can convince us to change asyncio: it's totally >> possible that the reasoning behind this restriction is no longer really >> valid. >> >> I expect that Yury will have to jump in, but I believe he's busy with a >> release. I also hope Nathaniel has something to say -- I wonder if trio >> supports nested event loops? (And maybe a Tornado developer?) >> > > Tornado does allow for nested event loops (or did, before we adopted > asyncio). It doesn't allow nested invocations of the *same* event loop. > Good to know. > One final thing. What we're talking about here is nested invocation of the >> "event pump". There's another form of nested event loop invocation where >> two separate event loop objects exist. That is a much more worrisome >> scenario, because callbacks associated with one event loop won't run at all >> while one is waiting for a task on the other loop. Fortunately that's not >> what is requested here. :-) >> > > I actually think that nesting multiple event loops is not so problematic, > or at least not so problematic to be worth explicitly prohibiting. You > wouldn't want to run_forever an inner event loop while an outer one is > blocked, but using an inner short-lived event loop is not so bad. It's not > good, because it does block the outer event loop, but there are plenty of > things you could do that do that - use requests instead of an async http > client, use an inner event loop from a different library that you can't > detect, etc. Why single out nesting one asyncio event loop inside another > as something to prohibit? > Hm, I didn't mean to single out nesting asyncio. According to (the extreme version of) asyncio's philosophy, *anything* that does I/O is a no-no. (Yes, some people feel even disk I/O should be done asynchronously, and there's a real implementation of that somewhere. Trio supports this: https://trio.readthedocs.io/en/latest/reference-io.html#asynchronous-filesystem-i-o .) > In the past, when I converted a django app to use tornado I went through a > phase where there were multiple nested IOLoops. First convert all the > outgoing network calls (which I guess were urllib2 at the time; requests > didn't exist yet) to spin up a short-lived IOLoop and run tornado's > AsyncHTTPClient (using the equivalent of IOLoop.run_sync, although that > method hadn't been added yet). Then replace the outer django handlers with > tornado handlers one at a time (using tornado's WSGIContainer to run the > django parts). Once WSGIContainer was gone, I could change all the run_sync > calls to yields so everything ran on the outer event loop. It wasn't the > prettiest or fastest thing I've ever done, but it worked. > I have to admit that practicality probably beats purity here. > As for jupyter, I think the best thing for them to do is run all notebook > user code in a separate thread dedicated to that purpose, and hide the fact > that the notebook itself is running asyncio as much as possible. That user > thread can start up its own event loop if it wants, but that's not the > jupyter kernel's concern. Until it can be refactored to use separate > threads, I think it would be reasonable to let it start up new event loops > (and run them for finite durations), although asyncio currently disallows > that as long as you're on the same thread as an outer event loop. > Given PBP, I wonder if we should just relent and have a configurable flag (off by default) to allow nested loop invocations (both the same loop and a different loop). -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
