potiuk opened a new pull request, #76139: URL: https://github.com/apache/airflow/pull/76139
Three triggerer tests run a `TriggerRunnerSupervisor` until it has handled its events and then leave its forked runner process running: - `test_trigger_can_call_variables_connections_and_xcoms_methods` - `test_trigger_can_fetch_trigger_dag_run_count_and_state_in_deferrable` - `test_trigger_can_fetch_dag_run_count_ti_count_in_deferrable` Since #76016 the triggerer freezes the GC before that fork and keeps it frozen, so in the pytest process the supervisor is never collected, its sockets are never closed, and the runner never sees EOF: it outlives the test. On Python 3.12.16, `ResourceTracker.__del__` closes the multiprocessing resource tracker's pipe and waits for the tracker at interpreter exit; the leaked runners inherited that pipe, so the tracker never exits and pytest hangs after printing its summary. That is why every Python 3.12 DB-core job has been timing out after the `Core` tests passed, on `main` and `v3-4-test` (#76135). The tests now stop and reap the runner after each run (`kill(force=True)` in a `finally`), which keeps #76016's memory behaviour unchanged. Verified locally with breeze on the Python 3.12.16 CI image (SQLite, DB tests): | Run (`executors/` + `jobs/test_triggerer_job.py`) | Result | |---|---| | `main` | hangs after `222 passed` | | `main` with #76016 reverted | exits (`220 passed`) | | this PR | exits (`222 passed`) | A `py-spy dump` of the hung session showed the main pytest process in `ResourceTracker.__del__` → `_stop_locked` and two leaked triggerer runner processes still in their asyncio loops. closes: #76135 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Opus 5 Generated-by: Claude Opus 5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
