RishabhBajaj97 opened a new pull request, #74650: URL: https://github.com/apache/airflow/pull/74650
When the metadata database has a transient disconnect (RDS/Aurora failover, idle-connection drop by the NLB, routine Postgres maintenance), the `UPDATE task_instance` inside `Trigger.clean_unused` raises `DBAPIError`. The `run_with_db_retries()` wrapper is supposed to absorb this, but it only matches `DBAPIError` / `StaleDataError` — and after a `DBAPIError`, the SQLAlchemy session is left in a pending-rollback state. The next retry attempt reuses that invalidated session and raises `PendingRollbackError`, which the retry policy does **not** match. The exception propagates out of `clean_unused`, the triggerer process exits, and every in-flight deferred trigger it was polling has to be reclaimed by another triggerer, losing its asyncio progress. The project already has the right pattern in `retry_db_transaction` (`airflow/utils/retries.py`), which catches `DBAPIError` / `StaleDataError` inside the retry attempt, calls `session.rollback()`, and re-raises so Tenacity drives the next retry on a clean session. `retry_db_transaction` can't be applied directly because its docstring says it must not be used with `@provide_session` (and `clean_unused` already is), so this PR mirrors the pattern inline. Validated: - New regression test `test_clean_unused_rolls_back_session_on_transient_db_error` passes with the fix (`1 passed in 10.20s`) and fails without it (`AssertionError: Expected 'rollback' to have been called once. Called 0 times.`). - Full `airflow-core/tests/unit/models/test_trigger.py` suite: 49 passed, no regressions. - `ruff format` + `ruff check` clean. - `mypy airflow-core/src/airflow/models/trigger.py` run inside Breeze: `Success: no issues found in 1 source file` (host-side mypy prek hook can't set up its venv on this machine due to a glibc mismatch unrelated to the diff). - Fast prek hooks (`prek run --from-ref main --stage pre-commit`) all pass for the matched files; `mypy-airflow-core` + `mypy-devel-common` skipped locally and verified via Breeze as noted above. Related: #70961 is an in-review PR that batches `clean_unused`; its restructure to `with create_session(scoped=False)` per retry attempt would incidentally close this issue as a side effect of the rework. This PR is a smaller, targeted fix in case the batching PR takes longer to land. closes: #74614 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 4.7) Generated-by: Claude Code (Opus 4.7) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
