Dev-iL opened a new pull request, #76694: URL: https://github.com/apache/airflow/pull/76694
related: https://github.com/apache/airflow/pull/75396#discussion_r4484303049, #76505 The async metadata engine never got the `isolation_level="READ COMMITTED"` that `prepare_engine_args()` sets for the sync engine on MySQL, so async sessions ran at the server default, `REPEATABLE READ`. The sync engine avoids that level because a query repeated in one transaction reads a stale snapshot. Task heartbeats (#67800) and audit log rows (#76416) already go through the async engine, and the XCom routes in #75396 will too. #76505 fixed the other half of the same gap, the session time zone. The rule now lives in one helper that both engines use, so the two can't drift again. PostgreSQL and SQLite engines get the same arguments as before. ## How it was shown working `test_async_engine_connections_get_the_sync_engine_setup` gains an `aiomysql` and an `asyncmy` case that read `@@session.transaction_isolation` through `settings.AsyncSession`. On `main` (`1fe65f5f13`), with only the new test cases: ```text breeze run --backend mysql bash -c "pytest airflow-core/tests/unit/core/test_settings.py -k test_async_engine_connections_get_the_sync_engine_setup" FAILED ...[mysql+aiomysql-SELECT @@session.transaction_isolation-READ-COMMITTED] - AssertionError: -'REPEATABLE-READ' +'READ-COMMITTED' FAILED ...[mysql+asyncmy-SELECT @@session.transaction_isolation-READ-COMMITTED] - AssertionError: -'REPEATABLE-READ' +'READ-COMMITTED' 2 failed, 2 passed, 1 skipped ``` With the fix, Breeze, Python 3.14.10: ```text breeze run --python 3.14 --backend <backend> bash -c "pytest airflow-core/tests/unit/core/test_settings.py" mysql: 71 passed, 4 skipped postgres: 67 passed, 8 skipped sqlite: 67 passed, 8 skipped ``` The same three runs gave the same results after rebasing onto `8930bc3db2`. The core test types that `breeze verify` lists for this change, run before committing: - **Non-DB suite:** `breeze testing core-tests --python 3.14 --use-xdist --skip-db-tests --backend none --parallel-test-types "API Always CLI Core Other Serialization"` gives 5603 passed, 2 failed both with and without this change. The 2 failures are `test_providers_manager` worker crashes from an illegal instruction on my machine, which lacks AVX2. - **DB suite:** the parallel DB run (`--run-db-tests-only`) fails at collection on my machine, identically on `main`. In its place, with MySQL as the backend (`breeze run --python 3.14 --backend mysql bash -c "pytest airflow-core/tests/unit/core airflow-core/tests/unit/utils/test_session.py airflow-core/tests/unit/utils/test_db.py airflow-core/tests/unit/utils/test_sqlalchemy.py airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_task_instances.py"`): 749 passed, 17 skipped. This covers the engine and session setup and the async heartbeat route. `prek run --stage pre-commit` on the changed files and `prek run mypy-airflow-core --all-files` pass. Debt left (found on the way, not changed here because each needs its own decision on what the async engine should honour): - `_configure_async_session()` copies the sync pool settings by hand instead of sharing them with `prepare_engine_args()`, and the copies already differ: the async engine ignores `configure_orm(disable_connection_pool=..., pool_class=...)` and `[database] sql_alchemy_engine_args`. - The `sql_alchemy_engine_args` description in `config.yml` doesn't say that it applies to the sync engine only, or that on MySQL a configured `isolation_level` is replaced with `READ COMMITTED`. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
