On Mon, May 11, 2015 at 1:37 PM, Paul Moore <p.f.mo...@gmail.com> wrote:
> On 6 May 2015 at 16:46, Guido van Rossum <gu...@python.org> wrote: > > This is actually a great idea, and I encourage you to go forward with it. > > The biggest piece missing from your inventory is probably Task, which is > > needed to wrap a Future around a coroutine. > > OK, I've been doing some work on this. You're right, the asyncio > framework makes Future a key component. > > But I'm not 100% sure why Future (and Task) have to be so fundamental. > Ignoring cancellation (see below!) I can build pretty much all of a > basic event loop, plus equivalents of the asyncio locks and queues > modules, without needing the concept of a Future at all. The > create_task function becomes simply a function to add a coroutine to > the ready queue, in this context. I can't return a Task (because I > haven't implemented the Task or Future classes) but I don't actually > know what significant functionality is lost as a result - is there a > reasonably accessible example of where using the return value from > create_task is important anywhere? > In asyncio the Task object is used to wait for the result. Of course if all you need is to wait for the result you don't need to call create_task() -- so in your situation it's uninteresting. But Task is needed for cancellation and Future is needed so I/O completion can be implemented using callback functions. > A slightly more complicated issue is with the run_until_complete > function, which takes a Future, and hence is fundamentally tied to the > Future API. However, it seems to me that a "minimal" implementation > could work by having a run_until_complete() that just took an > awaitable (i.e., anything that you can yield from). Again, is there a > specific reason that you ended up going with run_until_complete taking > a Future rather than just a coroutine? Actually it takes a Future *or* a coroutine. (The docs or the arg name may be confusing.) In asyncio, pretty much everything that takes one takes the other. > I think (but haven't confirmed > yet by implementing it) that it should be possible to create a > coroutine that acts like a Future, in the sense that you can tell it > from outside (via send()) that it's completed and set its return > value. But this is all theory, and if you have any practical > experience that shows I'm going down a dead end, I'd be glad to know. > I don't know -- I never explored that. > I'm not sure how useful this line of attack will be - if the API isn't > compatible with asyncio.BaseEventLoop, it's not very useful in > practice. On the other hand, if I can build a loop without Future or > Task classes, it may indicate that those classes aren't quite as > fundamental as asyncio makes them (which may allow some > simplifications or generalisations). > Have you tried to implement waiting for I/O yet? OTOH you may look at micropython's uasyncio -- IIRC it doesn't have Futures and it definitely has I/O waiting. > > I expect you'll also want to build cancellation into your "base async > > framework"; and the primitives to wait for multiple awaitables. The next > > step would be some mechanism to implement call_later()/call_at() (but > this > > needs to be pluggable since for a "real" event loop it needs to be > > implemented by the basic I/O selector). > > These are where I suspect I'll have the most trouble if I haven't got > a solid understanding of the role of the Future and Task classes (or > alternatively, how to avoid them :-)) So I'm holding off on worrying > about them for now. But certainly they need to be covered. In > particular, call_later/call_at are the only "generic" example of any > form of wait that actually *waits*, rather than returning immediately. > So as you say, implementing them will show how the basic mechanism can > be extended with a "real" selector (whether for I/O, or GUI events, or > whatever). > Right. > > If you can get this working it would be great to include this in the > stdlib > > as a separate "asynclib" library. The original asyncio library would > then be > > a specific implementation (using a subclass of asynclib.EventLoop) that > adds > > I/O, subprocesses, and integrates with the selectors module (or with > IOCP, > > on Windows). > > One thing I've not really considered in the above, is how a > refactoring like this would work. Ignoring the "let's try to remove > the Future class" approach above, my "basic event loop" is mostly just > an alternative implementation of an event loop (or maybe an > alternative policy - I'm not sure I understand the need for policies > yet). A policy is mostly a wrapper around an event loop factory plus state that records the current event loop. > So it may simply be a case of ripping coroutines.py, futures.py, > locks.py, log.py, queues.py, and tasks.py out of asyncio and adding a > new equivalent of events.py with my "minimal" loop in it. (So far, > when I've tried to do that I get hit with some form of circular import > problem - I've not worked out why yet, or how asyncio avoids the same > problem). > That sounds like a surface problem. Keep on debugging. :-) > That in itself would probably be a useful refactoring, splitting out > the IO aspects of asyncio from the event loop / async aspects. > Well, if you can. > > I don't see any particular hurry to get this in before 3.5; the > refactoring > > of asyncio can be done later, in a backward compatible way. It would be a > > good way to test the architecture of asyncio! > > Agreed. It's also not at all clear to me how the new async/await > syntax would fit in with this, so that probably needs some time to > settle down. For example, in Python 3.5 would run_until_complete take > an awaitable rather than a Future? > It doesn't need to change -- it already calls async() on its argument before doing anything (though with PEP 492 that function will be renamed to ensure_future()). -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
