I doubt if we should specify such things very explicitly. Call it "implementation detail" :)
FYI in Python 3.7 all `sock_*()` methods are native coroutines now. `run_in_executor()` is a regular function that returns a future object. I don't remember is it the only exception or asyncio has other functions with such return type. On Thu, May 3, 2018 at 11:56 PM Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > It would probably be hard for people to find at this point all the > places they might be relying on this behavior (if anywhere), but isn't > this a basic documented property of coroutines? > > From the introduction section on coroutines [1]: > > > Calling a coroutine does not start its code running – the coroutine > object returned by the call doesn’t do anything until you schedule its > execution. There are two basic ways to start it running: call await > coroutine or yield from coroutine from another coroutine (assuming the > other coroutine is already running!), or schedule its execution using the > ensure_future() function or the AbstractEventLoop.create_task() method. > > > Coroutines (and tasks) can only run when the event loop is running. > > [1]: https://docs.python.org/3/library/asyncio-task.html#coroutines > > --Chris > > On Thu, May 3, 2018 at 1:24 PM, Guido van Rossum <gvanros...@gmail.com> > wrote: > > Depending on the coroutine*not* running sounds like asking for trouble. > > > > On Thu, May 3, 2018, 09:38 Andrew Svetlov <andrew.svet...@gmail.com> > wrote: > >> > >> What real problem do you want to solve? > >> Correct code should always use `await loop.sock_connect(sock, addr)`, it > >> this case the behavior difference never hurts you. > >> > >> On Thu, May 3, 2018 at 7:04 PM twisteroid ambassador > >> <twisteroid.ambassa...@gmail.com> wrote: > >>> > >>> Hi, > >>> > >>> tl;dr: coroutine functions and regular functions returning Futures > >>> behave differently: the latter may start running immediately without > >>> being scheduled on a loop, or even with no loop running. This might be > >>> bad since the two are sometimes advertised to be interchangeable. > >>> > >>> > >>> I find that sometimes I want to construct a coroutine object, store it > >>> for some time, and run it later. Most times it works like one would > >>> expect: I call a coroutine function which gives me a coroutine object, > >>> I hold on to the coroutine object, I later await it or use > >>> loop.create_task(), asyncio.gather(), etc. on it, and only then it > >>> starts to run. > >>> > >>> However, I have found some cases where the "coroutine" starts running > >>> immediately. The first example is loop.run_in_executor(). I guess this > >>> is somewhat unsurprising since the passed function don't actually run > >>> in the event loop. Demonstrated below with strace and the interactive > >>> console: > >>> > >>> $ strace -e connect -f python3 > >>> Python 3.6.5 (default, Apr 4 2018, 15:01:18) > >>> [GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux > >>> Type "help", "copyright", "credits" or "license" for more information. > >>> >>> import asyncio > >>> >>> import socket > >>> >>> s = socket.socket() > >>> >>> loop = asyncio.get_event_loop() > >>> >>> coro = loop.sock_connect(s, ('127.0.0.1', 80)) > >>> >>> loop.run_until_complete(asyncio.sleep(1)) > >>> >>> task = loop.create_task(coro) > >>> >>> loop.run_until_complete(asyncio.sleep(1)) > >>> connect(3, {sa_family=AF_INET, sin_port=htons(80), > >>> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection > >>> refused) > >>> >>> s.close() > >>> >>> s = socket.socket() > >>> >>> coro2 = loop.run_in_executor(None, s.connect, ('127.0.0.1', 80)) > >>> strace: Process 13739 attached > >>> >>> [pid 13739] connect(3, {sa_family=AF_INET, sin_port=htons(80), > >>> >>> sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED > (Connection refused) > >>> > >>> >>> coro2 > >>> <Future pending cb=[_chain_future.<locals>._call_check_cancel() at > >>> /usr/lib64/python3.6/asyncio/futures.py:403]> > >>> >>> loop.run_until_complete(asyncio.sleep(1)) > >>> >>> coro2 > >>> <Future finished exception=ConnectionRefusedError(111, 'Connection > >>> refused')> > >>> >>> > >>> > >>> Note that with loop.sock_connect(), the connect syscall is only run > >>> after loop.create_task() is called on the coroutine AND the loop is > >>> running. On the other hand, as soon as loop.run_in_executor() is > >>> called on socket.connect, the connect syscall gets called, without the > >>> event loop running at all. > >>> > >>> Another such case is with Python 3.4.2, where even loop.sock_connect() > >>> will run immediately: > >>> > >>> $ strace -e connect -f python3 > >>> Python 3.4.2 (default, Oct 8 2014, 10:45:20) > >>> [GCC 4.9.1] on linux > >>> Type "help", "copyright", "credits" or "license" for more information. > >>> >>> import socket > >>> >>> import asyncio > >>> >>> loop = asyncio.get_event_loop() > >>> >>> s = socket.socket() > >>> >>> c = loop.sock_connect(s, ('127.0.0.1', 82)) > >>> connect(7, {sa_family=AF_INET, sin_port=htons(82), > >>> sin_addr=inet_addr("127.0.0.1")}, 16) = -1ECONNREFUSED (Connection > >>> refused) > >>> >>> c > >>> <Future finished exception=ConnectionRefusedError(111, 'Connection > >>> refused')> > >>> >>> > >>> > >>> In both these cases, the misbehaving "coroutine" aren't actually > >>> defined as coroutine functions, but regular functions returning a > >>> Future, which is probably why they don't act like coroutines. However, > >>> coroutine functions and regular functions returning Futures are often > >>> used interchangeably: Python docs Section 18.5.3.1 even says: > >>> > >>> > Note: In this documentation, some methods are documented as > coroutines, > >>> > even if they are plain Python functions returning a Future. This is > >>> > intentional to have a freedom of tweaking the implementation of these > >>> > functions in the future. > >>> > >>> In particular, both run_in_executor() and sock_connect() are > >>> documented as coroutines. > >>> > >>> If an asyncio API may change from a function returning Future to a > >>> coroutine function and vice versa any time, then one cannot rely on > >>> the behavior of creating the "coroutine object" not running the > >>> coroutine immediately. This seems like an important Gotcha waiting to > >>> bite someone. > >>> > >>> Back to the scenario in the beginning. If I want to write a function > >>> that takes coroutine objects and schedule them to run later, and some > >>> coroutine objects turn out to be misbehaving like above, then they > >>> will run too early. To avoid this, I could either 1. pass the > >>> coroutine functions and their arguments separately "callback style", > >>> 2. use functools.partial or lambdas, or 3. always pass in real > >>> coroutine objects returned from coroutine functions defined with > >>> "async def". Does this sound right? > >>> > >>> Thanks, > >>> > >>> twistero > >>> _______________________________________________ > >>> Async-sig mailing list > >>> Async-sig@python.org > >>> https://mail.python.org/mailman/listinfo/async-sig > >>> Code of Conduct: https://www.python.org/psf/codeofconduct/ > >> > >> -- > >> Thanks, > >> Andrew Svetlov > >> _______________________________________________ > >> Async-sig mailing list > >> Async-sig@python.org > >> https://mail.python.org/mailman/listinfo/async-sig > >> Code of Conduct: https://www.python.org/psf/codeofconduct/ > > > > > > _______________________________________________ > > Async-sig mailing list > > Async-sig@python.org > > https://mail.python.org/mailman/listinfo/async-sig > > Code of Conduct: https://www.python.org/psf/codeofconduct/ > > > -- Thanks, Andrew Svetlov
_______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/