jason810496 opened a new issue, #76903: URL: https://github.com/apache/airflow/issues/76903
### Description Task loop rules are checked in two places, by two copies of the code: - **Python Dags**: the Task SDK checks loops while the Dag is written, in `task-sdk/src/airflow/sdk/definitions/_internal/loop.py` and `DAG.validate()`. - **Language SDK Dags** (TypeScript, Go, Java): the Dag processor checks the serialized Dag, in `airflow-core/src/airflow/dag_processing/loop_checks.py` (added in #76803). The rules are meant to be the same, but they already differ in one place: `check_expansions_over_loop_tasks` is copied nearly line for line, and the terminal-task check is not the same. Python skips teardowns when it picks the terminal task, as `TaskGroup.get_leaves()` does; core does not. Move the rules that look at the whole Dag into one module in `shared/dagnode`. It takes the graph as callbacks, as `detect_cycle` does, so both `DAG.validate()` and `loop_checks.py` can call it: - No `expand` over a loop task from outside the loop. - No Dag result inside a loop. - No nested loop, and no loop inside a mapped task group. - The loop body ends in exactly one terminal task, skipping teardowns. These stay where they are: - Checks that run while a Python Dag is written, such as those in `create_loop`. They point the author at the line that is wrong. - Core's gate checks (the gate is an unmapped `LoopGateOperator` and gates its loop). Python builds the gate itself, so only language SDK Dags need them. Done when: - Each rule above has one implementation, used by both paths, with the same error message. - A language SDK loop whose body ends in a teardown gets the same terminal task Python picks. - Python Dags parse as they do today. ### Use case/motivation ADR-0017 decision 3 makes the Dag processor the authority on loop rules for language SDKs, while Python keeps its own checks. With two copies, a rule fixed in one place can be missed in the other, and the same loop can then be accepted from one language and rejected from another. ### Related issues - #76803 adds the Dag processor checks for language SDK Dags. - ADR-0017: `airflow-core/adr/lang-sdk/0017-task-loops-across-lang-sdks.md` ### Are you willing to submit a PR? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
