[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Yury Selivanov]

Yury Selivanov  added the comment:

> I suppose that it would also be difficult to get buy-in for a feature like 
> this from the different frameworks?

Maybe :)  Ideally, asyncio programs should not depend on how exactly tasks are 
scheduled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Vitaly Kruglikov]

Vitaly Kruglikov  added the comment:

Yet another paradigm is the likes of `GSource` in gnome's glib. GSource "tasks" 
added to the event loop are polled by the event loop for readiness before the 
event loop blocks on select/epoll/etc.. The ones that are ready are removed 
from the loop and their callbacks are dispatched.

I suppose that it would also be difficult to get buy-in for a feature like this 
from the different frameworks?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Vitaly Kruglikov]

Vitaly Kruglikov  added the comment:

I'll have to settle for `set_write_buffer_limits(0, 0)` for my blocking wrapper 
case.

I think that 'write_buffer_drained' callback is a good idea, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Vitaly Kruglikov]

Vitaly Kruglikov  added the comment:

> 'events.AbstractEventLoop.run_one_step()'

> This is highly unlikely to ever happen.

Sure, I can see how that could be a problem with other underlying 
implementations, such as Twisted reactor. Just wishful thinking :).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Yury Selivanov]

Yury Selivanov  added the comment:

> 'events.AbstractEventLoop.run_one_step()'

This is highly unlikely to ever happen.  It's hard to define what one iteration 
of an event loop is, and it would be even harder to get that agreement for all 
frameworks/event loops that are compatible with or based on asyncio.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Yury Selivanov]

Yury Selivanov  added the comment:

Yeah, I think your best option would be to use `set_write_buffer_limits(0, 0)`. 
 You don't need asyncio flow control anyways, as AMQP protocol is unlikely to 
generate any pressure on IO buffers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Vitaly Kruglikov]

Vitaly Kruglikov  added the comment:

... or `events.AbstractEventLoop.run_one_iteration()`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Vitaly Kruglikov]

Vitaly Kruglikov  added the comment:

Thank you for following up. My use case is this:

In the Pika AMQP client package, we have a blocking AMQP connection adapter - 
`BlockingConnection` - wrapped around an asynchronous AMQP connection adapter. 
Presently, BlockingConnection is wrapped around the asynchronous 
`SelectConnection` which has a home-grown proprietary IOLoop. I would like to 
switch BlockingConnection to use an asyncio-based adapter when running on 
Python3.

SelectConnection's proprietary IOLoop provides a single-stepping run function 
(poll with a timeout as normally determined by pending callbacks, timers, etc., 
process all ready events/timers/callbacks, and return). When BlockingConnection 
needs to send something to the AMQP broker and/or wait for an expected reply, 
it sends the request (which typically gets queued up in a write buffer) and 
then runs the proprietary IOLoop's single-stepping method in a loop (thus 
blocking efficiently on I/O); each time after the single-stepping IOLoop method 
returns, BlockingConnection checks whether the conditions of interest have been 
satisfied (e.g., the write buffer size being zero and AMQP Channel.OpenOk has 
been received).

So, another way that asyncio could help, and certainly simplest for me and my 
use case, is by providing a single-stepping function in the event loop 
implementations, such as the single-stepping method that I described at the top 
of the previous paragraph. E.g., `events.AbstractEventLoop.run_one_step()`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Yury Selivanov]

Yury Selivanov  added the comment:

We'll likely add 'write_buffer_drained' callback method to `asyncio.Protocol` 
in 3.8.  In the meanwhile, the only option would be using `_make_empty_waiter` 
in 3.7, or set_write_buffer_limits(0, 0).

What's your use case, by the way?

--
versions:  -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33118] No clean way to get notified when a Transport's write buffer empties out

[Vitaly Kruglikov]

New submission from Vitaly Kruglikov :

There doesn't appear to be an ordained mechanism for getting notified when a 
Transport's (or WriteTransport's) write buffer drains to zero (i.e., all output 
data has been transferred to socket). I don't want to hijack 
`set_write_buffer_limits()` for this purpose, because that would preclude me 
from using it for its intended purpose.

I see that transport in selector_events.py has a private method 
`_make_empty_waiter()`, which is along the lines of what I need, but it's 
private and is used by `BaseSelectorEventLoop._sendfile_native()`.

Just like `BaseSelectorEventLoop._sendfile_native()`, my app needs equivalent 
functionality in order to be able to run the loop (`run_until_complete()`) 
until the transport's write buffer empties out.

--
components: asyncio
messages: 314236
nosy: asvetlov, vitaly.krug, yselivanov
priority: normal
severity: normal
status: open
title: No clean way to get notified when a Transport's write buffer empties out
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com