jason810496 opened a new pull request, #69015:
URL: https://github.com/apache/airflow/pull/69015
## Wh
Python 3.14 changed the Unix `multiprocessing` default from `fork` to
`forkserver`, which makes the Celery worker's stdlib multiprocessing helpers
re-import Airflow and spawn extra forkserver/resource-tracker processes,
inflating resident memory.
## What
- Add `[celery] mp_start_method` and `[celery] mp_forkserver_preload` config
options (fall back to `[core]` equivalents).
- `airflow celery worker` now calls
`set_component_mp_start_method("celery")` before spawning any stdlib helper
(serve_logs, stale-bundle cleanup, SecretCache Manager), guarded behind
`AIRFLOW_V_3_3_PLUS`.
- [Billiard's](https://github.com/celery/billiard) prefork pool keeps its
own `fork` default and is unaffected.
- Add a test asserting the worker applies the configured start method.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes, with help of Claude Code (Opus 4.8) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
