[issue37736] asyncio.wait_for is still confusing

[Andrew Borba]

Andrew Borba  added the comment:

> The key word here is "misbehaving". Cooperative concurrency does require 
> cooperation.

Your stance makes sense in an ideal environment where we control every Task 
that our application deals with. However, there are some cases in which busted 
Task cancellation logic is out of our control to fix, and in some cases it may 
be critical to strictly enforce a timeout regardless of what a Task does. If 
wait_for is only to be used as a best effort, then we need a nuclear option as 
well.

--
nosy: +andrewborba

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37736] asyncio.wait_for is still confusing

[Caleb Hattingh]

Caleb Hattingh  added the comment:

> asyncio.wait_for is still confusing

Perhaps the confusion can be fixed with improvements to the docs? To me, these 
specific docs seem pretty clear now, but I might not be a good judge of that.

> However, we still have the case where a misbehaving Task can cause wait_for 
> to hang indefinitely.

The key word here is "misbehaving". Cooperative concurrency does require 
cooperation. There are many ways in which coroutines can misbehave, the popular 
one being calling blocking functions when they shouldn't. I would be very 
uncomfortable with my coroutine being killable (e.g. by wait_for) by some other 
means besides CancelledError (which I can intercept and manage cleanup).  

The contract is: if my coroutine has a CancelledError raised, I take that to 
mean that I need to clean up whatever resources need cleanup, in a timely 
manner and then exit. If my coro refuses to exit, it is my coroutine that is 
wrong, not wait_for being unable to kill the coroutine.

I definitely agree with Yury that the previous behaviour, the one where 
wait_for could raise TimeoutError *before* the inner coro has exited, was buggy 
and needed to be fixed.

--
nosy: +cjrh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37736] asyncio.wait_for is still confusing

[David Lewis]

New submission from David Lewis :

This issue is a follow up to previous discussions about confusing results with 
asyncio.wait_for. In the current implementation, it seems unintuitive that a 
coroutine with a timeout argument may easily wait forever. Perhaps wait_for 
could use an await_cancellation=True kwarg.

Prior issues:

a) "It's a feature, not a bug" - Guido
https://github.com/python/asyncio/issues/253#issuecomment-120020018

b) "I don't feel comfortable with asyncio living with this bug till 3.8." - Yury
https://bugs.python.org/issue32751#msg318065

Originally, wait_for would cancel the future and raise TimeoutError 
immediately. In the case of a Task, it could remain active for some time after 
the timeout, since its cancellation is potentially asynchronous.

In (a), this behaviour was defended, since waiting on the cancellation would 
violate the implicit contract of the timeout argument to wait_for(). While the 
documentation suggests it's a poor idea, it's not illegal for a task to defer 
or entirely refuse cancellation.

In (b), the task outliving the TimeoutError was considered a bug, and the 
behaviour changed to its current state. To address the issue raised in (a), the 
documentation for wait_for now contains the line "The function will wait until 
the future is actually cancelled, so the total wait time may exceed the 
timeout."

However, we still have the case where a misbehaving Task can cause wait_for to 
hang indefinitely. For example, the following program doesn't terminate:

import asyncio, contextlib

async def bad():
while True:
with contextlib.suppress(asyncio.CancelledError):
await asyncio.sleep(1)
print("running...")

if __name__ == '__main__':
asyncio.run(asyncio.wait_for(bad(), 1))


More realistically, the task may have cooperative cancellation logic that waits 
for something else to be tidied up:

try:
await wait_for(some_task(service), 10)
except TimeoutError:
   ...
finally:
service.stop()

--
components: asyncio
messages: 348848
nosy: David Lewis, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wait_for is still confusing
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com