> On Oct 20, 2025, at 1:11 PM, Kirill Miazine <[email protected]> wrote:
>
> 2025-10-20T21:45:53+0200 [pyng.master.PyngMaster#info] Unable to format event
> {'check': <pyng.PING o
> bject at 0x294df13f3510>, 'log_logger': <Logger 'pyng.master.PyngMaster'>,
> 'log_level': <LogLevel=in
> fo>, 'log_namespace': 'pyng.master.PyngMaster', 'log_source':
> <pyng.master.PyngMaster object at 0x29
> 4df34c8090>, 'log_format': 'Ready to run {check.type} check "{check.desc}"
> [{check.id <http://check.id/>}]', 'log_time'
> : 1760989553.4371765}: maximum recursion depth exceeded
My guess is that what's happening here is that because logging consumes several
stack frames for the emitting, the formatting, etc, you're just getting unlucky
and happening to blow your stack here as it's the biggest increase in the
repeated function. The message here is telling you that it can't format the
message, but given that PyngCheck does not seem to have a dynamic __getattr__
or any @property descriptors, I don't think that that's where you're seeing the
recursion.
The actual problem that I think you're seeing is just too large a number of
Deferreds synchronously running their callbacks (which you can see via the
addCallbacks on the top of the stack synchronously resulting in a _runCallbacks
call. You're creating a Deferred for each go round the loop, without actually
returning back up to the event loop at any point, eventually building up a huge
stack of Deferred-calling-Deferred-calling-Deferred. IIRC we have some
mitigations for this to reduce the amount of recursion that Deferred does but
it is not avoidable in the general case.
I suspect that the thing that you need here is just some backoff and jitter on
some part of the retry here, rather than synchronously attempting to do things?
I don't have time to fully absorb this logic but I can definitely see some
mutual synchronous recursion between _get_check and _run_check, which is, I
think, where you are running into problems (specifically with the implicit
callback addition caused by returning a Deferred from _run_check into
_get_check's addCallback).
Even if your stack is not meaningfully infinite, it's easy to get up around the
stack limit if you just have a huge pile of Deferreds synchronously chain on to
one another like this. But even a deferLater(0) will break up the compounding
callback stack and allow work to proceed.
-g
_______________________________________________
Twisted mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/twisted.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/RZKF6UB35BDCEHARQTRLRXPRMBS4ZPGR/
Code of Conduct: https://twisted.org/conduct
