Andrushika commented on PR #74869:
URL: https://github.com/apache/airflow/pull/74869#issuecomment-6261620383
FYI @ChickenBenny, this graph helps me understand the architecture between
coordinators, executors, and the task runner a lot. I hope this can help you as
well :)
```mermaid
flowchart LR
subgraph SP["Scheduler OS process"]
subgraph LE["LocalExecutor object"]
AQ["activity_queue<br/>IPC"]
RQ["result_queue<br/>IPC"]
WH["workers<br/>Process handles"]
end
end
subgraph WP["Local worker OS process｜可重用"]
LOOP["_run_worker()<br/>get loop"]
CM["CoordinatorManager.for_queue()<br/>[sdk] queue_to_coordinator"]
PYC["_PythonCoordinator<br/>queue 沒對應時的 fallback"]
LSC["SubprocessCoordinator<br/>Executable／Java／Node"]
subgraph SV["Supervisor｜supervise_task()"]
SUP["ActivitySubprocess object<br/>WatchedSubprocess subtype"]
PSUP["_PopenActivitySubprocess object<br/>ActivitySubprocess
subtype"]
end
end
subgraph TP["Task Runner child process｜每個 Python task 一個"]
TR["task_runner.main()<br/>operator user code"]
end
subgraph LP["Lang runtime process｜每個 lang SDK task 一個"]
GO["Go: bundle binary<br/>BundleRef.Serve()"]
JV["Java: java -classpath ...<br/>Server.serve()"]
TS["TS: node bundle.mjs<br/>bundle.serve()"]
end
AQ ==>|worker gets workload| LOOP
LOOP -->|supervise_task per task| CM
CM -->|queue not mapped| PYC
CM -->|queue mapped| LSC
PYC -.->|creates| SUP
LSC -.->|creates| PSUP
SUP ==>|puts outcome| RQ
PSUP ==>|puts outcome| RQ
WH -->|tracks／joins／terminates| LOOP
SUP -->|forks／starts + waits| TR
PSUP -->|launches runtime + waits| GO & JV & TS
classDef focus fill:#fff3bf,stroke:#e67700,stroke-width:3px
class SUP,PSUP focus
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
