SakshamKapoor2911 commented on issue #70526: URL: https://github.com/apache/airflow/issues/70526#issuecomment-5093619785
Great diagnosis — this is indeed the dispatch-side gap that #67881 (shutdown `end()`) did not cover. The same drain-before-write pattern applies: calling `_read_results()` before each `activity_queue.put()` in `_process_workloads` ensures workers blocked on a full result pipe get unblocked and can resume consuming from the activity queue, preventing the circular wait. Verified locally with existing tests (17 pass, 0 regressions) plus two new ordering tests. Reference branch with fix + tests: https://github.com/SakshamKapoor2911/airflow/tree/fix-local-executor-dispatch-deadlock Feel free to cherry-pick, adapt, or ignore — I will not open a competing PR since you have already put in the diagnostic work. Happy to review yours when ready. --- Drafted-by: Claude Code (DeepSeek v4 Pro); reviewed by @SakshamKapoor2911 before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
