On Wed, Feb 19, 2014 at 10:20 PM, Nikolaus Rath <[email protected]> wrote:
> Guido van Rossum <[email protected]> writes: > > Coroutines can only yield asyncio Futures. And yes, that's an internal > > implementation detail. > > > > However, you probably should look into integrating with the event loop > > using callbacks instead of coroutines. Callbacks are totally vanilla > > functions, they just shouldn't actually ever block for I/O -- when a > > callback wants to do some blocking I/O it should use an event loop > > callback, probably using add_reader/add_writer to wait until the I/O is > > ready. (The latter won't work with an IOCP event loop on Windows, but > then > > again, your example uses select() directly, so you probably don't care. > :-) > > > > OTOH maybe I am totally misunderstanding what you are looking for -- I > > haven't managed to grok your docs. > > (Unrelated to the main question, but is there some specific part I could > improve in the docs?) > I didn't try very hard. > I think you answered my question, and the answer is "No, it won't > work". But let me rephrase it, just to make sure :-). > > Suppose somebody is using asyncio, and we're in the middle of some > asyncio coroutine: > > @asyncio.coroutine > def do_stuff(b) > #... > yield from asyncio.sleep(1) > # ... > yield from other_fn() > # ... > > Is there something that I could yield in other_fn that will cause it to > be resumed once a specific fd is writable? > If other_fn() was a regular function it could return a Future. Or if it was an asyncio coroutine (decorated with @asyncio.coroutine) it could yield from a Future. However it can't just yield a Future -- that will make a boobietrap go off specifically set up to prevent you from making the mistake of yielding a Future instead of yielding *from* it. Then of course you have to arrange for the Future's result to be set when your FD is ready. You do that with an I/O callback. I'll explain below. > I'm thinking about something like this: > > def other_fn(): > # ... > yield asyncio.resume_me_when_writable(some_fd) > buf = os.write(some_fd, 42) > # ... > > The add_writer() method of the BaseEventLoop seems related to this, but > doesn't quite do what I want. I don't want to register a callback, I > want to resume the coroutine.. > Yes, you do want to register a write callback that calls f.set_result(None) (and the probably removes the write callback. I think if you study the implementation of sock_sendall() (and _sock_sendall()) in asyncio/selector_events.py you'll get my gist; sock_sendall() is a function returning a Future that will be made complete when the I/O is done; _sock_sendall() first just tries to send (all FDs here are nonblocking) and if that fails it registers itself as a write callback and then upon being called again it unregisters itself and tries to send again (hoping that this time it will not block, otherwise it'll just re-register itself and so on). Good luck! > > > Best, > -Nikolaus > > > > > > > On Wed, Feb 19, 2014 at 6:40 PM, Nikolaus Rath < > [email protected]> wrote: > > > >> Hello, > >> > >> I have used coroutines in a non-blocking HTTP client module for some > >> time. I have now finally cleaned and documented the code enough for it > >> to be useful to other people (no official release on PyPI yet, but code > >> and documentation is available on > >> https://bitbucket.org/nikratio/python-dugong and > >> http://pythonhosted.org/dugong/). > >> > >> I think it would be rather nice if I could make this module work with > >> asyncio as well, but without having to depend on it. > >> > >> I looked over the asyncio documentation at > >> http://docs.python.org/3.4/library/asyncio.html, but somehow I can't > >> seem to find the right information. > >> > >> Currently, the coroutines in my module run until they encounter an I/O > >> operation that would block. In this case, they yield a tuple that > >> contains a file descriptor and the type of I/O operation they would like > >> to perform. For example, to implement a blocking read function I would > >> do: > >> > >> coroutine = conn._co_read(256) # Method from my module > >> try: > >> while True: > >> io_req = next(coroutine) > >> assert io_req.mask == select.EPOLLIN > >> select.select((io_req.fd,), (), (), 0) > >> except StopIteration as exc: > >> buf = exc.value > >> > >> > >> Now, asyncio seems to support coroutines as well, but I can't find any > >> information about what my coroutine is expected to yield in order for it > >> to work with an asyncio event loop. > >> > >> I suppose that information isn't that often needed, because you are > >> supposed to use e.g. asyncio.StreamReader, in which case you can just > >> use "yield from". However, this isn't really an option for code that is > >> supposed to *integrate* with asyncio rather than *depend* on it. > >> > >> > >> Is the type of objects that are yielded by coroutines considered an > >> implementation detail that I should not depend on, or is it just not > >> documented yet? In the former case: is there any other way for me to > >> make my module compatible? > >> > >> > >> Thanks, > >> -Nikolaus > >> > >> -- > >> Encrypted emails preferred. > >> PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C > >> > >> »Time flies like an arrow, fruit flies like a Banana.« > >> > > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > > > -- > Encrypted emails preferred. > PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C > > »Time flies like an arrow, fruit flies like a Banana.« > -- --Guido van Rossum (python.org/~guido)
