nmatravolgyi <nandor.ma...@gmail.com> added the comment:

I did a little research on how the `wait_for` could be improved. There are four 
alternate implementations you can find here: 

The repository (on the linked branch) has tox set-up and a test that asserts 
the behaviour of builtin and alternate implementations.

Quick summary:
  - wait_for (builtin): either leaks resources or ignores cancellation
  - wait_for (preferred alt): works as expected (I'm going to discuss this 
  - wait_for_special_raise: works, but has pitfalls
  - wait_for_no_waiter*: These were proposed by @aaliddell on the related PR: 
https://github.com/python/cpython/pull/26097#issuecomment-840497455 I could not 
make them work, but I might be missing something. I think the fact that the 
inner coroutine gets wrapped into a Future introduces the race-condition, but I 
don't know how to test that. The general idea of this would be the best if an 
implementation was possible.

About the actually working alternate implementation I made:
In my opinion there is no way to implicitly handle losing a result properly in 
case of a cancellation, since it arbitrarily breaks the flow of execution. I'd 
look into having `wait_for(...)` support cleanup callbacks when a cancellation 
and completion happens at the same time. Something like:


async def create_something():
    # this acquires some resource that needs explicit cleanup after some work
    return object()

def cleanup_something(inst):

t = asyncio.ensure_future(create_something())
x = await asyncio.wait_for(t, timeout=10, cancel_handler=cleanup_something)
    pass  # normal work with x
    cleanup_something(x)  # cleanup at normal execution

The inner task is still responsible for handling the resource before it 
returns, which means if the inner task is cancelled, there must be no leak if 
the implementation is correct. If no cancellation happens, everything is 
"fine". Finally, the waiter task would be able to handle its cancellation and 
the simultaneous completion of the inner task if the caller code provides a 

Unfortunately this requires the user of `wait_for` to be aware of this 
race-condition. However it makes it possible to be handled properly when the 
waited future's result requires a cleanup.

nosy: +nmatravolgyi

Python tracker <rep...@bugs.python.org>
Python-bugs-list mailing list

Reply via email to