gang-zh commented on PR #71723: URL: https://github.com/apache/airflow/pull/71723#issuecomment-5432889600
We've been running this exact change in production (patched into our image, equivalent to `_should_use_exec()` returning true) on Airflow 3.2.2 / KubernetesExecutor. Data points: - **Repro:** a TLS-stress probe in a real task pod deadlocked **16 of 13,930 bare-forked children** (all parked in `pthread_rwlock_wrlock` under `SSL_CTX_new_ex`, matching #71707) vs **0 of 5,195 with fork+exec**, same pod, same stress. - **Overhead:** measured **+8.8s median per task** on a plugin-heavy image — essentially the child re-importing the SDK. Note it lands in *task duration*, not queued duration: the TI is marked running right after the fork, before the new interpreter boots. - **Production:** two deployments running with exec enabled; no hangs since, deferral resume / retries / XCom all normal. Before this, we accumulated dozens of tasks silently hung for days — `execution_timeout` never fires since the hang precedes `execute()`, as the issue says. The opt-in config is the right shape given the overhead. Longer term, forking before the supervisor spawns threads (or a forkserver-style template) could give the same safety without the re-import cost — but that's a redesign; this is the right bridge. +1 to land, and a v3-2/v3-3 backport would be very welcome — every KubernetesExecutor user on those lines forks cold per task. Happy to test a candidate build or share the probe script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
