ephraimbuddy opened a new pull request, #58935: URL: https://github.com/apache/airflow/pull/58935
When two schedulers run concurrently, both could start more backfill dag runs than max_active_runs allows. This happened because each scheduler read the count of running dag runs before either committed, causing both to see stale counts and start runs simultaneously. The fix adds row-level locking on the Backfill table. When a scheduler processes backfill dag runs, it first locks the relevant Backfill rows. If another scheduler already holds the lock, the current scheduler skips those backfills rather than potentially violating the max_active_runs constraint. This ensures that only one scheduler can process a given backfill's dag runs at a time, preventing the race condition while remaining non-blocking (schedulers don't wait on each other). (cherry picked from commit 22af27ea5a750ed92b203bd47846e5197cfda7fa) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
