What real problem do you want to solve? Correct code should always use `await loop.sock_connect(sock, addr)`, it this case the behavior difference never hurts you.
On Thu, May 3, 2018 at 7:04 PM twisteroid ambassador < [email protected]> wrote: > Hi, > > tl;dr: coroutine functions and regular functions returning Futures > behave differently: the latter may start running immediately without > being scheduled on a loop, or even with no loop running. This might be > bad since the two are sometimes advertised to be interchangeable. > > > I find that sometimes I want to construct a coroutine object, store it > for some time, and run it later. Most times it works like one would > expect: I call a coroutine function which gives me a coroutine object, > I hold on to the coroutine object, I later await it or use > loop.create_task(), asyncio.gather(), etc. on it, and only then it > starts to run. > > However, I have found some cases where the "coroutine" starts running > immediately. The first example is loop.run_in_executor(). I guess this > is somewhat unsurprising since the passed function don't actually run > in the event loop. Demonstrated below with strace and the interactive > console: > > $ strace -e connect -f python3 > Python 3.6.5 (default, Apr 4 2018, 15:01:18) > [GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import asyncio > >>> import socket > >>> s = socket.socket() > >>> loop = asyncio.get_event_loop() > >>> coro = loop.sock_connect(s, ('127.0.0.1', 80)) > >>> loop.run_until_complete(asyncio.sleep(1)) > >>> task = loop.create_task(coro) > >>> loop.run_until_complete(asyncio.sleep(1)) > connect(3, {sa_family=AF_INET, sin_port=htons(80), > sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection > refused) > >>> s.close() > >>> s = socket.socket() > >>> coro2 = loop.run_in_executor(None, s.connect, ('127.0.0.1', 80)) > strace: Process 13739 attached > >>> [pid 13739] connect(3, {sa_family=AF_INET, sin_port=htons(80), > sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused) > > >>> coro2 > <Future pending cb=[_chain_future.<locals>._call_check_cancel() at > /usr/lib64/python3.6/asyncio/futures.py:403]> > >>> loop.run_until_complete(asyncio.sleep(1)) > >>> coro2 > <Future finished exception=ConnectionRefusedError(111, 'Connection > refused')> > >>> > > Note that with loop.sock_connect(), the connect syscall is only run > after loop.create_task() is called on the coroutine AND the loop is > running. On the other hand, as soon as loop.run_in_executor() is > called on socket.connect, the connect syscall gets called, without the > event loop running at all. > > Another such case is with Python 3.4.2, where even loop.sock_connect() > will run immediately: > > $ strace -e connect -f python3 > Python 3.4.2 (default, Oct 8 2014, 10:45:20) > [GCC 4.9.1] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import socket > >>> import asyncio > >>> loop = asyncio.get_event_loop() > >>> s = socket.socket() > >>> c = loop.sock_connect(s, ('127.0.0.1', 82)) > connect(7, {sa_family=AF_INET, sin_port=htons(82), > sin_addr=inet_addr("127.0.0.1")}, 16) = -1ECONNREFUSED (Connection > refused) > >>> c > <Future finished exception=ConnectionRefusedError(111, 'Connection > refused')> > >>> > > In both these cases, the misbehaving "coroutine" aren't actually > defined as coroutine functions, but regular functions returning a > Future, which is probably why they don't act like coroutines. However, > coroutine functions and regular functions returning Futures are often > used interchangeably: Python docs Section 18.5.3.1 even says: > > > Note: In this documentation, some methods are documented as coroutines, > even if they are plain Python functions returning a Future. This is > intentional to have a freedom of tweaking the implementation of these > functions in the future. > > In particular, both run_in_executor() and sock_connect() are > documented as coroutines. > > If an asyncio API may change from a function returning Future to a > coroutine function and vice versa any time, then one cannot rely on > the behavior of creating the "coroutine object" not running the > coroutine immediately. This seems like an important Gotcha waiting to > bite someone. > > Back to the scenario in the beginning. If I want to write a function > that takes coroutine objects and schedule them to run later, and some > coroutine objects turn out to be misbehaving like above, then they > will run too early. To avoid this, I could either 1. pass the > coroutine functions and their arguments separately "callback style", > 2. use functools.partial or lambdas, or 3. always pass in real > coroutine objects returned from coroutine functions defined with > "async def". Does this sound right? > > Thanks, > > twistero > _______________________________________________ > Async-sig mailing list > [email protected] > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ > -- Thanks, Andrew Svetlov
_______________________________________________ Async-sig mailing list [email protected] https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
