potiuk opened a new issue, #76135: URL: https://github.com/apache/airflow/issues/76135
### What happens In the full-matrix CI runs, the `Core` test type in every **Python 3.12** DB-core job runs all of its tests successfully, prints the pytest summary, and then the process never exits. Breeze waits until the total test timeout, terminates the pool, and the job fails with exit code 2: ``` 3834 passed, 48 skipped, 968 deselected, 1 xfailed, 1 warning in 1224.07s (0:20:24) ... Terminating the pool All containers stopped. Marking the whole run as terminated on timeout Some tasks were terminated on timeout. Please check the logs of the tasks (below) for more details. The core test DB failed! Giving up ``` There is no failing test and no traceback. The other test types in the same job (`API`, `Always`, `CLI`, `Other`, `Serialization`) report OK. Example timeline (Postgres 15 / 3.12, v3-4-test): `Core` started 07:45:47, printed its summary at 08:06:28, and was idle until the pool was killed at 08:42:20. ### Scope - Python 3.12 only: the same jobs on 3.11, 3.13 and 3.14 pass in the same runs. - All DB backends: SQLite, Postgres 15, MySQL 8.4 and LowestDeps all fail this way. - Both `main` and `v3-4-test`. Regular `main` push runs only test the default Python, so they stay green; the nightly full matrix shows it. - Every failing job runs **Python 3.12.16** (`platform linux -- Python 3.12.16, pytest-9.1.1`). - First seen on `v3-4-test` between the nightly of 2026-10-29 16:46 (green, `dfa73fddf0`) and the nightly of 2026-10-30 04:46 (red, `410177f3b2`). The five commits in that range (UI, docs and e2e only) do not touch core code or the test setup, which points at the rebuilt CI image rather than the code. ### Examples - `main` nightly: https://github.com/apache/airflow/actions/runs/41157855924/job/124410276636 (Sqlite 3.12) - `v3-4-test`: https://github.com/apache/airflow/actions/runs/41113132731/job/124480035970 (Postgres 15 / 3.12) - `v3-4-test` nightly: https://github.com/apache/airflow/actions/runs/41093082487 (all four 3.12 DB-core jobs) ### Likely cause and next steps Most likely something in the `Core` test session (a non-daemon thread, a multiprocessing child, or an atexit handler) keeps the interpreter from shutting down under the Python 3.12 version in the new image. To find out what is blocking: 1. Dump the stacks at session end, e.g. arm `faulthandler.dump_traceback_later(...)` in `pytest_unconfigure`, so a CI run shows what blocks shutdown. 2. Reproduce locally with `breeze testing core-tests --test-type Core --backend sqlite --python 3.12` and run `py-spy dump` on the hung process. 3. Compare against the previous 3.12 patch release by pinning the base image. --- Drafted-by: Claude Code (Opus 5.5); reviewed by @potiuk before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
