tirkarthi commented on PR #64882:
URL: https://github.com/apache/airflow/pull/64882#issuecomment-4204672915
> In order to make sure that we only send+receive one message at a time I
have
swapped from the `threading.lock` to `aiologic.Lock` which allows for
better
locking/waiting behaviour between threads (which is what Triggers will
use via
the `sync_to_async`) and the main TriggerRunner.
We were also affected by this in 3.1.5 and while exploring the previous code
I came across da78b396f4c788e914fb33c10e79568e1db4a312 which seems related and
had above comment. Is using aiologic based lock a better option here?
One of our major worries was also that triggerer kept heartbeating though it
was processing no triggers as reported in the issue due to this issue. Hence
the liveness probe was not restarting the triggerer process. Is there something
else to be done to fix the issue? We had deadlock in our mysql cluster from
time to time as we use end from trigger feature which lead to triggerer
restarts on deadlock and sort of worked around this issue.
https://pypi.org/project/aiologic/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
