The GitHub Actions job "Tests (AMD)" on airflow.git/fix-celery-publish-fork-deadlock has failed. Run started by GitHub user dheerajturaga (triggered by dheerajturaga).
Head commit for run: bd75de174b96be88eb12e2380eb2b836c96c738e / Dheeraj Turaga <[email protected]> Address review: reuse Celery state-fetch pool and keep fork escape hatch The bulk state-fetch pool is rebuilt on every sync. Its forkserver and spawn workers re-import Airflow rather than inheriting it, so a fresh pool per heartbeat pays that import cost every time, which the rpc and mongodb result backends hit on every scheduler loop. Reuse one pool for the fetcher's lifetime and shut it down when the executor ends; recreate it if a worker death breaks it so one dead worker cannot wedge fetching. Resolving the start method was gated on Airflow 3.3+, so on the 2.11 through 3.2 versions this provider still supports, the pinned mp_start_method config was silently ignored and operators had no way to ask for fork back. Read the config directly so the escape hatch works everywhere. Report URL: https://github.com/apache/airflow/actions/runs/30488391527 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
