novain-shibu opened a new pull request, #74574: URL: https://github.com/apache/airflow/pull/74574
--- When one task group feeds another (`group_a >> group_b`), `dag_edges` (used by `/ui/structure/structure_data`, i.e. the Graph view) called `get_roots()` on the downstream group once for every leaf of the upstream group. `get_roots()` and `get_leaves()` walk the whole group on each call, so building the edges grew roughly cubically with group size. This change computes each group's root and leaf ids once and reuses them. The edges returned are unchanged. Timing of `dag_edges` alone on a serialized Dag with two groups of N/2 parallel tasks (`g1 >> g2`), Airflow 3.3.2, GitHub Actions runner: | Tasks | Before | After | |---|---|---| | 400 | 0.22 s | 0.06 s | | 800 | 1.59 s | 0.28 s | | 1600 | 10.34 s | 1.15 s | What remains grows with the number of real task dependencies between the two groups ((N/2)² here), which the Dag itself defines. Checks: - Output identical to the current code on 64 Dags: 8 shapes (chains, fan-out, group chains, wide groups, many groups feeding one group, nested groups, setup/teardown chains) plus 40 random mixed Dags with nested groups, setup/teardown and task-to-group edges. - New test `test_dag_edges_computes_group_roots_and_leaves_once`; it fails without the change. - `test_task_group.py` and `test_edgemodifier.py` pass; ruff check/format clean. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Claude Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) Claude Code found the slowdown with a scaling probe, wrote the change and the test, and ran the checks. I reviewed every line and can explain the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
