michaelmicheal commented on code in PR #30079:
URL: https://github.com/apache/airflow/pull/30079#discussion_r1137254779
##########
airflow/dag_processing/manager.py:
##########
@@ -679,6 +680,10 @@ def _run_parsing_loop(self):
@provide_session
def _fetch_callbacks(self, max_callbacks: int, session: Session =
NEW_SESSION):
+ self._fetch_callbacks_with_retries(max_callbacks, session)
+
+ @retry_db_transaction
+ def _fetch_callbacks_with_retries(self, max_callbacks: int, session:
Session):
"""Fetches callbacks from database and add them to the internal queue
for execution."""
self.log.debug("Fetching callbacks from the database.")
with prohibit_commit(session) as guard:
Review Comment:
Wouldn't we want to retry upon any failure related to the guard, or any
queries within the guard? I'm not especially opinionated, but this seems like
the safest option to ensure a failure doesn't crash a standalone dag-processor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
