I’m also +1 on 2a) On Thu 24 Sep 2026 at 23:46, Damian Shaw <[email protected]> wrote:
> Anecdotally as a user who was impacted by this, it was pretty annoying and > had it been intentional it would have been very frustrating to not have a > deprecation notice. > > That said, it was also trivial to fix (with LLM assistance) as, for me, it > was simply caused by implicit edges that had to be made explicit. > > I'm a big +1 for warning users before upgrading, and let users know what > if they call "check_cycle(dag)" in their CI for exceptions, they should > also call something like "group.topological_sort() for group in > dag.task_group.get_task_group_dict().values()" in CI for exceptions. > > Damian > > -----Original Message----- > From: Jarek Potiuk <[email protected]> > Sent: Thursday, September 24, 2026 5:16 PM > To: [email protected] > Subject: Re: [DISCUSS] Handling cyclic TaskGroup dependencies (Grid/Graph > 500 since 3.3.1) > > I also lean towards 2a) - with rejection in 3.5 - especially that task > group loop behaviour is not going to land in 3.4 it seems. > > And my proposals to answer the questions: > > > 1. Should Dags with group-level cycles be allowed at all, long term? > > Generally - I think no. Task Group - when loops are implemented, will > invalidate all those kinds of dependencies. In fact **any** dependency for > tasks in Task Group going out (except the "exit" of the TaskGroup) should > be deprecated in 3.4 and removed in 3.5. > However - I think we should survey or query a large number of users' Dags > to see the potential impact. And we should revisit this in case we find > that it will break a lot of Dags. > One option if we find this is a "common" scenario will be to add a Dag or > "per airflow instance" option to either allow this or Task Group loops. > > 2. If we decide to disallow them, how should we roll that out? Some > options: > a) Fix the Grid/Graph 500 in 3.4.x and emit a deprecation / Dag warning > at parse time. Reject in 3.5. > > I would also treat it as a bug that it was allowed before. I don't think > we ever intended such "task groups" behaviours. > > 3. If we decide to allow them, is a fallback ordering in Grid/Graph (as in > #72822) acceptable? How should the planned TaskGroup features handle these > Dags? > > As above. Ideally not allowing is "cleanest" but we should consider > migration pain if we have some indication this pattern is common. And > either ease migration or have a flag to disable Task Group loops. > > 4. Communication: should we add a known-issue note to the 3.3.1/3.3.2 > release notes, and publish upgrade guidance on how to restructure affected > Dags? > > I would even say that this might be a reason to have 3.3.3 and fix it > there (and add deprecation there as well) - if we decide to re-enable it > (with > deprecation) in 3.4.0. > > J. > > On Thu, Sep 24, 2026 at 8:42 PM Blain David <[email protected]> > wrote: > > > Hi Dheeraj, > > > > Thanks for bringing this to the list, I remember this issue being > > mentioned in the previous devcall ;-) > > > > I'd lean towards option 2a. > > > > If TaskGroups were only a visual grouping, I'd say the fix belongs in > > the UI alone: the task graph is a valid DAG, so Grid and Graph should > > just tolerate the group-level cycle and pick a fallback order, as #72822 > does. > > Rendering an imperfect order is better than a 500 on a Dag that runs > fine. > > > > But from earlier discussions my understanding is that TaskGroups are > > meant to become part of the execution logic (loops, dynamic groups, > > clearing or retrying a group, waiting on a group) in the future? Those > > features need an unambiguous group order, so I agree these Dags should > > eventually be rejected. > > That doesn't have to be a parse error right away though. I'd propose: > > > > 1. > > Short term (3.3.x / 3.4.x): fix the 500 in Grid/Graph with a fallback > > ordering, and raise a Dag warning at parse time naming the TaskGroups > > involved and stating that group-level cycles are deprecated. A Dag > > warning is visible in the UI, which is where affected users will > > actually notice it. Their Dags keep parsing and running. > > 2. > > Later: turn the warning into an import error. Given our deprecation > > policy that's a breaking change, so realistically 4.0 rather than a > > 3.x minor, but this is just speculation and thinking out loud. In the > > meantime the planned TaskGroup features can refuse to operate on a > > group that is part of a cycle, without affecting plain scheduling. > > > > Just my thoughts. > > > > Kind regards, > > David > > > > ________________________________ > > From: Dheeraj Turaga <[email protected]> > > Sent: Thursday, September 24, 2026 20:00 > > To: [email protected] <[email protected]> > > Subject: [DISCUSS] Handling cyclic TaskGroup dependencies (Grid/Graph > > 500 since 3.3.1) > > > > [You don't often get email from [email protected]. Learn why this is > > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze > > niet vertrouwt, klik niet op een link of open geen bijlages. Bij > > twijfel, stuur deze e-mail als bijlage naar [email protected]<mailto: > > [email protected]>. > > > > Hi all, > > > > I'd like to start a discussion about a regression introduced in > > Airflow 3.3.1. The fix we choose will be a breaking change for some > > users either way. > > I'm affected by this on my own Dags too, so I'd like us to agree on a > > direction before anything lands. > > > > Tracking issue: > > https://github.com/apache/airflow/issues/73678 > > <https://github.com/apache/airflow/issues/73678> > > > > WHAT HAPPENED > > > > Since 3.3.1, the Grid and Graph views return HTTP 500 for Dags whose > > TaskGroup dependencies form a cycle when each group is treated as a > > single unit. > > At the task level these Dags are acyclic, and they parse, schedule and > > run normally. > > > > A minimal example: > > > > with TaskGroup("group1"): > > a1 = EmptyOperator(task_id="a1") > > a2 = EmptyOperator(task_id="a2") > > with TaskGroup("group2"): > > b1 = EmptyOperator(task_id="b1") > > b2 = EmptyOperator(task_id="b2") > > > > a1 >> b1 # group1 -> group2 > > b2 >> a2 # group2 -> group1 > > > > A simpler variant also triggers it: two tasks in the same group > > connected through a task outside the group (a >> ext >> b). > > > > This is common when TaskGroups are used for logical or visual grouping > > (for example one group per database schema). Users have hit it when > > upgrading from 2.x to 3.3.1. > > > > The cause is a bug fix. Grid/Graph used to ignore group-to-group and > > cross-group edges when ordering TaskGroups, and silently rendered an > > arbitrary order. The fix made the sort account for those edges, so > > group-level cycles now raise an error: > > - > > https://github.com/apache/airflow/pull/69933 > > <https://github.com/apache/airflow/pull/69933> > > - > > https://github.com/apache/airflow/pull/70591 > > <https://github.com/apache/airflow/pull/70591> (backport, released in > > 3.3.1) > > > > APPROACHES SO FAR > > > > - > > https://github.com/apache/airflow/pull/72822 > > <https://github.com/apache/airflow/pull/72822> (closed): keep these > > Dags valid and make Grid/Graph fall back to a > > strongly-connected-component ordering. > > - > > https://github.com/apache/airflow/pull/73087 > > <https://github.com/apache/airflow/pull/73087> (open): reject these > > Dags at parse time with an import error that names the TaskGroups > involved. > > > > The arguments on each side, from the PR discussions: > > > > - For treating them as invalid: planned TaskGroup features such as > > task loops, dynamic TaskGroups, retrying or clearing a TaskGroup, and > > waiting for a TaskGroup to complete need an unambiguous group > > ordering. They would have undefined behaviour on these Dags. > > - For allowing them: the task graph itself is valid. Many users use > > TaskGroups only for grouping. Rejecting these Dags in a patch release > > would turn a UI-only problem into Dags failing to load, which > > conflicts with our deprecation policy ( > > > > https://airflow.apache.org/docs/apache-airflow/stable/release-process. > > html#deprecation-policy > > < > > https://airflow.apache.org/docs/apache-airflow/stable/release-process. > > html#deprecation-policy > > > > > ). > > > > QUESTIONS > > > > 1. Should Dags with group-level cycles be allowed at all, long term? > > > > 2. If we decide to disallow them, how should we roll that out? Some > > options: > > a) Fix the Grid/Graph 500 in 3.4.x and emit a deprecation / Dag > > warning at parse time. Reject in a later minor or major release. > > b) Deprecate it starting 3.4.0 > > > > 3. If we decide to allow them, is a fallback ordering in Grid/Graph > > (as in > > #72822) acceptable? How should the planned TaskGroup features handle > > these Dags? > > > > 4. Communication: should we add a known-issue note to the 3.3.1/3.3.2 > > release notes, and publish upgrade guidance on how to restructure > > affected Dags? > > > > > > Thanks, > > Dheeraj > > > > General (Internal Property) > > > ________________________________ > Strike Technologies, LLC (“Strike”) is part of the GTS corporate family. > Strike is a technology solutions provider, and is not a broker or dealer > and does not engage in securities transactions. This communication does not > constitute an offer to sell or the solicitation of an offer to buy any > security in any jurisdiction.. > ________________________________ > > CONFIDENTIALITY / PRIVILEGE NOTICE: This communication and any attachments > are intended solely for the addressee. The information contained in this > communication may be proprietary, privileged, confidential, and/or > protected from unauthorized use or disclosure under applicable law. If you > are not the intended recipient, you are hereby notified that any review, > dissemination, distribution, copying, or other use or disclosure of this > information is strictly prohibited, and may be unlawful. If you have > received this transmission in error, please delete this message and all > copies from your system and notify the sender via return transmittal. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >
