I also lean towards 2a) - with rejection in 3.5 - especially that task group loop behaviour is not going to land in 3.4 it seems.
And my proposals to answer the questions: 1. Should Dags with group-level cycles be allowed at all, long term? Generally - I think no. Task Group - when loops are implemented, will invalidate all those kinds of dependencies. In fact **any** dependency for tasks in Task Group going out (except the "exit" of the TaskGroup) should be deprecated in 3.4 and removed in 3.5. However - I think we should survey or query a large number of users' Dags to see the potential impact. And we should revisit this in case we find that it will break a lot of Dags. One option if we find this is a "common" scenario will be to add a Dag or "per airflow instance" option to either allow this or Task Group loops. 2. If we decide to disallow them, how should we roll that out? Some options: a) Fix the Grid/Graph 500 in 3.4.x and emit a deprecation / Dag warning at parse time. Reject in 3.5. I would also treat it as a bug that it was allowed before. I don't think we ever intended such "task groups" behaviours. 3. If we decide to allow them, is a fallback ordering in Grid/Graph (as in #72822) acceptable? How should the planned TaskGroup features handle these Dags? As above. Ideally not allowing is "cleanest" but we should consider migration pain if we have some indication this pattern is common. And either ease migration or have a flag to disable Task Group loops. 4. Communication: should we add a known-issue note to the 3.3.1/3.3.2 release notes, and publish upgrade guidance on how to restructure affected Dags? I would even say that this might be a reason to have 3.3.3 and fix it there (and add deprecation there as well) - if we decide to re-enable it (with deprecation) in 3.4.0. J. On Thu, Sep 24, 2026 at 8:42 PM Blain David <[email protected]> wrote: > Hi Dheeraj, > > Thanks for bringing this to the list, I remember this issue being > mentioned in the previous devcall ;-) > > I'd lean towards option 2a. > > If TaskGroups were only a visual grouping, I'd say the fix belongs in the > UI alone: the task graph is a valid DAG, so Grid and Graph should just > tolerate the group-level cycle and pick a fallback order, as #72822 does. > Rendering an imperfect order is better than a 500 on a Dag that runs fine. > > But from earlier discussions my understanding is that TaskGroups are meant > to become part of the execution logic (loops, dynamic groups, clearing or > retrying a group, waiting on a group) in the future? Those features need > an unambiguous group order, so I agree these Dags should eventually be > rejected. > That doesn't have to be a parse error right away though. I'd propose: > > 1. > Short term (3.3.x / 3.4.x): fix the 500 in Grid/Graph with a fallback > ordering, and raise a Dag warning at parse time naming the TaskGroups > involved and stating that group-level cycles are deprecated. A Dag warning > is visible in the UI, which is where affected users will actually notice > it. Their Dags keep parsing and running. > 2. > Later: turn the warning into an import error. Given our deprecation policy > that's a breaking change, so realistically 4.0 rather than a 3.x minor, but > this is just speculation and thinking out loud. In the meantime the planned > TaskGroup features can refuse to operate on a group that is part of a > cycle, without affecting plain scheduling. > > Just my thoughts. > > Kind regards, > David > > ________________________________ > From: Dheeraj Turaga <[email protected]> > Sent: Thursday, September 24, 2026 20:00 > To: [email protected] <[email protected]> > Subject: [DISCUSS] Handling cyclic TaskGroup dependencies (Grid/Graph 500 > since 3.3.1) > > [You don't often get email from [email protected]. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze > niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel, > stuur deze e-mail als bijlage naar [email protected]<mailto: > [email protected]>. > > Hi all, > > I'd like to start a discussion about a regression introduced in Airflow > 3.3.1. The fix we choose will be a breaking change for some users either > way. > I'm affected by this on my own Dags too, so I'd like us to agree on a > direction before anything lands. > > Tracking issue: > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fissues%2F73678&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cf518208e80c8416044e208df1a65d593%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639258696828902551%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=UIWub0tb7vdQ1un%2FKLFX4TL61Aj%2Bz5WjzMEOvx%2Be0Lk%3D&reserved=0 > <https://github.com/apache/airflow/issues/73678> > > WHAT HAPPENED > > Since 3.3.1, the Grid and Graph views return HTTP 500 for Dags whose > TaskGroup dependencies form a cycle when each group is treated as a single > unit. > At the task level these Dags are acyclic, and they parse, schedule and run > normally. > > A minimal example: > > with TaskGroup("group1"): > a1 = EmptyOperator(task_id="a1") > a2 = EmptyOperator(task_id="a2") > with TaskGroup("group2"): > b1 = EmptyOperator(task_id="b1") > b2 = EmptyOperator(task_id="b2") > > a1 >> b1 # group1 -> group2 > b2 >> a2 # group2 -> group1 > > A simpler variant also triggers it: two tasks in the same group connected > through a task outside the group (a >> ext >> b). > > This is common when TaskGroups are used for logical or visual grouping (for > example one group per database schema). Users have hit it when upgrading > from 2.x to 3.3.1. > > The cause is a bug fix. Grid/Graph used to ignore group-to-group and > cross-group edges when ordering TaskGroups, and silently rendered an > arbitrary order. The fix made the sort account for those edges, so > group-level cycles now raise an error: > - > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F69933&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cf518208e80c8416044e208df1a65d593%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639258696828923099%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=50iga%2FLUjFdFW7PdjdnIjeiEJMHgzG2ficU4JY%2Fv16w%3D&reserved=0 > <https://github.com/apache/airflow/pull/69933> > - > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F70591&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cf518208e80c8416044e208df1a65d593%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639258696828938243%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=oiDDTdyS%2Fve8u%2Bo2%2F%2FOfeVQ01dRkMIFuZf%2B3hw9hqSA%3D&reserved=0 > <https://github.com/apache/airflow/pull/70591> (backport, released in > 3.3.1) > > APPROACHES SO FAR > > - > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F72822&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cf518208e80c8416044e208df1a65d593%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639258696828951112%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=11GzICgfUQIZP3Xz0XjHO4ksIV1vOWJ5q3WlelYVtD0%3D&reserved=0 > <https://github.com/apache/airflow/pull/72822> (closed): keep these Dags > valid and make Grid/Graph fall back to a strongly-connected-component > ordering. > - > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F73087&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cf518208e80c8416044e208df1a65d593%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639258696828962690%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Cd4mhRVOLuNsKt%2F24rGCpKSZVNvDkt%2B0ypwwG3OmNmk%3D&reserved=0 > <https://github.com/apache/airflow/pull/73087> (open): reject these Dags > at parse time with an import error that names the TaskGroups involved. > > The arguments on each side, from the PR discussions: > > - For treating them as invalid: planned TaskGroup features such as task > loops, dynamic TaskGroups, retrying or clearing a TaskGroup, and waiting > for a TaskGroup to complete need an unambiguous group ordering. They would > have undefined behaviour on these Dags. > - For allowing them: the task graph itself is valid. Many users use > TaskGroups only for grouping. Rejecting these Dags in a patch release would > turn a UI-only problem into Dags failing to load, which conflicts with our > deprecation policy ( > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fdocs%2Fapache-airflow%2Fstable%2Frelease-process.html%23deprecation-policy&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cf518208e80c8416044e208df1a65d593%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639258696828974226%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gVhaaKboYPDh7V1O%2B6w8DMCdLN6bEkSY%2B13AloPXL1M%3D&reserved=0 > < > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#deprecation-policy > > > ). > > QUESTIONS > > 1. Should Dags with group-level cycles be allowed at all, long term? > > 2. If we decide to disallow them, how should we roll that out? Some > options: > a) Fix the Grid/Graph 500 in 3.4.x and emit a deprecation / Dag warning > at parse time. Reject in a later minor or major release. > b) Deprecate it starting 3.4.0 > > 3. If we decide to allow them, is a fallback ordering in Grid/Graph (as in > #72822) acceptable? How should the planned TaskGroup features handle these > Dags? > > 4. Communication: should we add a known-issue note to the 3.3.1/3.3.2 > release notes, and publish upgrade guidance on how to restructure affected > Dags? > > > Thanks, > Dheeraj > > General (Internal Property) >
