schizophrenicmaniac commented on issue #73583: URL: https://github.com/apache/airflow/issues/73583#issuecomment-5797776307
I’d like to work on this and submit a PR. #### Diagnosis The `py-spy` dump points to a post-fork deadlock: OpenTelemetry's `_after_in_child` fork hook (`TracerProvider._handle_fork`) calls `_get_process_dependent_resource()`, which spins up a `concurrent.futures.ThreadPoolExecutor` inside an `at_fork` handler while inheriting locked/corrupted threading state from the multi-threaded parent process. Furthermore, forked children inherit dead `BatchSpanProcessor` daemon threads, causing subsequent `force_flush` calls to stall. #### Proposed Fix 1. **Patch `TracerProvider._handle_fork`:** Neutralize `TracerProvider._handle_fork` in `airflow_shared.observability.traces` to prevent it from invoking resource detection tasks on a `ThreadPoolExecutor` inside the fork hook. 2. **Post-Fork Reset:** Register an `os.register_at_fork(after_in_child=...)` hook (mirroring `metrics.stats`) to reset `trace._TRACER_PROVIDER` to a no-op proxy provider in forked children, eliminating dead daemon threads and flush stalls. 3. **Verification:** Add tests verifying that `LocalExecutor` workers and DAG processor children start and complete cleanly when OpenTelemetry is enabled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
