[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I am still not convinced that audit events should be used.

Maybe support of explicit callbacks pair (on_start() + on_finish()) with `None` 
for fast-and-cheap "do nothing flag" is a better alternative for catching stale 
coroutines?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Tom Forbes]

Tom Forbes  added the comment:

Actually reacting to a stall would require something more and probably should 
be done at some point.

But this is purely about monitoring - in our use case we'd send a metric via 
statsd that would be used to correlate stalls against other service level 
metrics. This seems pretty critical when running a large number of asyncio 
applications in production because you can only currently _infer_ that a stall 
is happening, and it's hard to trace the cause across service boundaries. An 
event hook that was sent the loop and handle would be ideal for this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Steve Dower]

Steve Dower  added the comment:

Fundamentally I don't have an issue with the audit hook. My only concern would 
be if there's anything that an application may do to _respond_ to a stall (e.g. 
is this valuable for applying backpressure? etc.)

If it's purely diagnostic, and there's nothing you'd do in production when it 
happens, then an audit hook is perfect.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Tom Forbes]

Change by Tom Forbes :


--
keywords: +patch
pull_requests: +24642
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25990

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Tom Forbes]

Tom Forbes  added the comment:

I don't see why we shouldn't use PEP 578 for this - the events provide rich 
monitoring information about what a Python process is "doing" with an easy, 
central way to register callbacks to receive these events and shovel them off 
to a monitoring solution.

Is there that much of a difference between monitoring the number of files, 
sockets, emails or even web browsers opened and the number of times an asyncio 
application has stalled?

The alternative would be to make the loop stalling some kind of hookable event, 
which just seems like reinventing `sys.audit()`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Christian Heimes]

Christian Heimes  added the comment:

Are you proposing to use PEP 578 for monitoring the event loop?

--
nosy: +christian.heimes, steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44075] Add a PEP578 audit hook for Asyncio loop stalls

[Tom Forbes]

New submission from Tom Forbes :

Detecting and monitoring loop stalls in a production asyncio application is 
more difficult than it could be.

Firstly you must enable debug mode for the entire loop then you need to look 
for warnings outputted via the asyncio logger. This makes it hard to send loop 
stalls to monitoring systems via something like statsd.

Ideally asyncio callbacks would always be timed and an auditevent always 
triggered if it passes a particular threshold. If debug mode is enabled then a 
warning is logged.

--
components: asyncio
messages: 393251
nosy: asvetlov, orf, yselivanov
priority: normal
severity: normal
status: open
title: Add a PEP578 audit hook for Asyncio loop stalls
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com