boqiang commented on issue #59120: URL: https://github.com/apache/airflow/issues/59120#issuecomment-6177170379
I've prepared a scoped fix and regression tests in https://github.com/apache/airflow/pull/74761. The change lets `DBAPIError` escape both serialized-Dag loading and scheduled DagRun creation, so the existing transaction owner can roll back and retry the batch, re-querying candidates and reacquiring their row locks. `CommitProhibitorGuard` and ordinary per-Dag exception handling stay unchanged. The MySQL regression uses two real InnoDB transactions to trigger error 1213 after a DagRun has already been created in the batch. It passes with autoflush both enabled and disabled, verifies a fresh locked candidate query, exactly one run per scheduled Dag, and preservation of the concurrent metadata update. Reverting the production change makes all six new MySQL cases fail with `PendingRollbackError` following the original database error. The complete MySQL scheduler test file passes (566 passed, 2 skipped). This addresses recovery from transient database errors. Persistent errors still propagate when the configured retry limit is exhausted, so the broader per-Dag isolation work in this issue remains open. The tests exercise the transaction boundary directly; they do not establish multi-scheduler process acceptance. The PR includes the broader validation results and remaining limitations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
