github-actions[bot] opened a new pull request, #71858: URL: https://github.com/apache/airflow/pull/71858
* Move task execution architecture docs to contributing-docs The "Task execution architecture" section added in #69750 documents worker internals (the Supervisor, the Coordinator layer, and the language subprocess protocols). That is developer-facing material, not something end users need in the core-concepts overview. Relocate the whole section and its diagrams to a new dev-facing page under contributing-docs, next to the new-language-SDK guide, and drop it from the user-facing overview. Follow-up to review feedback on #70657. * Add task execution architecture section to Architecture Overview The overview explained how components are deployed but not what happens inside a worker when a task actually runs. This adds a concise section and diagram distinguishing the supervisor process, the CoordinatorManager that routes a task to a coordinator by its queue, and the language SDK subprocess launched for non-Python tasks, so readers can reason about the process boundaries. The full developer-level detail remains in contributing-docs. * Simplify worker task execution diagram in Architecture Overview Deployment Managers reading the Architecture Overview need to know what runs on a worker while tasks execute, so they can size workers for the memory and start-up cost each concurrent task instance adds. Coordinator routing, the msgpack wire protocol and per-runtime internals do not serve that question and already live in the developer guide. The previous wording also implied Python tasks run inside the supervisor with no separate process, which is not what happens: every task instance gets its own subprocess whatever language it is written in. * Highlight task execution lifecycle documentation (cherry picked from commit bfb121a2ee5e3b25e670d07b6b192c17eab67eac) Co-authored-by: Jason(Zhe-You) Liu <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
