YoannAbriel opened a new pull request, #63600:
URL: https://github.com/apache/airflow/pull/63600
## Problem
Asset-triggered DAGs get duplicate DAG runs when running with multiple
schedulers (HA mode). A single asset event creates two runs with identical
timestamps but different run ID hashes. Reported consistently within 15-30
minutes of running 2 schedulers.
## Root Cause
`DagModel.dags_needing_dagruns()` reads `AssetDagRunQueue` records without
row-level locking. Both schedulers read the same ADRQ rows, both evaluate the
DAG as ready, both create a DAG run, and both delete the ADRQ records. The
DagModel rows are locked with `FOR UPDATE SKIP LOCKED` but the ADRQ rows that
feed into the decision are not.
## Fix
Added `with_row_locks(skip_locked=True, of=AssetDagRunQueue)` to the ADRQ
select query. When scheduler A locks an ADRQ row, scheduler B skips it and
won't see the DAG as ready — preventing duplicate runs.
Added a test verifying the ADRQ query uses row-level locking.
Closes: #63507
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 4, claude-opus-4-6)
Generated-by: Claude Code (Opus 4, claude-opus-4-6) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
