Thanks for your feedback Przemysław, I updated the AIP On Fri, 7 Aug 2026 at 03:20, Przemysław Mirowski <[email protected]> wrote:
> Hi Kaxil, > > I've added some comments to the AIP. In general, the idea looks > interesting. > > Could we possibly rearrange the sections a bit? During the reading, > sometimes one thing was stated when the reader does not know a particular > part of the change (e.g. merge policy - I added a comment next to it). > Also, there are a lot of new definitions - maybe it would be good to create > a section which will gather them all and shortly explain? > > On 2026/07/30 10:16:20 Kaxil Naik wrote: > > Hi all, > > > > I would like to start a discussion on AIP-113: Dynamic Task Groups. > > > > > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440304823/AIP-113+Dynamic+Task+Groups > > > > tldr: > > Dynamic Task Mapping made cardinality dynamic: run the same task over a > > collection discovered at run time. What is still static is the shape, > > meaning which tasks > > run in a region and how they depend on each other. > > > > Branching gets you half of that already, and I want to be precise about > > which half, because > > I had this wrong in an earlier draft. `choose_branch` can return a list, > so > > a branch already > > selects an arbitrary subset of its declared successors. What a branch > > cannot do is decide > > the dependency edges *among* the tasks it selected. Selection is solved; > > ordering and wiring are not. That is the > > gap this AIP is aimed at. > > > > The proposal is a Task Group whose members and internal edges are decided > > at run time from a fixed catalog the author declares. A generator task > > emits a plan, the group materializes > > exactly the selected entries as real task instances, and the region > > converges on a > > synthesized merge node. The catalog is what serializes, so two runs of > > different shapes share > > one DAG version, and an entry the plan did not select never becomes a > task > > instance. The > > planner can be a control table, an impact analysis, a triage rule, or an > > LLM. > > > > --- > > > > Would love to hear your thoughts & get feedback. Please comment on the > AIP > > doc itself. > > > > Regards, > > Kaxil > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
