1fanwang opened a new pull request, #66788: URL: https://github.com/apache/airflow/pull/66788
When more than one Dag processor writes the same brand-new Dag in the same scheduler heartbeat, only one of them wins the INSERT into `serialized_dag` / `dag` / `dag_version`. The losers raise a unique-constraint `IntegrityError` (`UNIQUE constraint failed` on SQLite, `Duplicate entry` / 1062 on MySQL, `violates unique constraint` on Postgres), which `_serialize_dag_capturing_errors` currently routes through the generic `except Exception` arm. That has two user-visible effects in the same parsing cycle: 1. The Dag file ends up in `import_errors` even though the winner produced the correct row, so the UI flags a healthy file as broken until the next cycle. 2. The session is left in an aborted state, so subsequent per-Dag writes in the same `update_dag_parsing_results_in_db` call raise `PendingRollbackError`. The winner already produced the right row, so the loser's failure is a no-op rather than an import error. Add an `except IntegrityError` arm before the generic catch that: - Detects unique-constraint violations across SQLite, MySQL, and Postgres using the same message-prefix approach `_UniqueConstraintErrorHandler` already uses in the public API. - Rolls the session back so the rest of the cycle can proceed. - Logs at INFO and returns an empty import-error list (no retry — the row is there). Non-unique IntegrityErrors (e.g. NOT-NULL violations from a genuinely malformed Dag) still flow into the generic arm and surface as import errors, so users keep seeing real serialization problems. Tests cover the three dialect message shapes and the NOT-NULL fall-through case. Closes: #66786 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes - Claude Code (Opus 4.7) Generated-by: Claude Code (Opus 4.7) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
