Planner Tasks sounds good.

We already described mapped tasks as "dynamic" so using that word again would 
only confuse people.

On 2026/11/20 12:31:44 Amogh Desai via dev wrote:
> Thanks for proposing this. I like the idea, and catalog + validator seems
> like a reasonable
> way to keep the LLM planner bounded.
> 
> I left a few comments on the AIP.
> 
> Thanks & Regards,
> Amogh Desai
> 
> 
> On Thu, Nov 19, 2026 at 12:03 AM Kaxil Naik <[email protected]> wrote:
> 
> > I do like Planner Tasks
> >
> > On Wed, 18 Nov 2026 at 17:56, Constance Martineau via dev <
> > [email protected]> wrote:
> >
> > > For the name, I'm find myself defaulting to calling the proposed
> > > feature Planner Tasks, especially when discussing it alongside Mapped
> > > Tasks and Task Loops. Plan mode doesn't fit as well with the other
> > > concepts we have.
> > >
> > > On Wed, Nov 18, 2026 at 12:49 PM Kaxil Naik <[email protected]> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Picking this thread back up.
> > > >
> > > > Airflow 3.4 shipped Task Loops and YAML Dags, and both made AIP-113
> > > > simpler, so Ash and I reworked it on top of them: selected tasks are
> > now
> > > > created the way loop iterations are, and the plan uses the YAML Dag
> > > schema.
> > > > The page is updated:
> > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440304823/AIP-113+Dynamic+Task+Groups
> > > >
> > > > Thanks Przemysław for the earlier comments.
> > > >
> > > > Why now: the Common AI provider reached 1.0 in October, and more of the
> > > > Dags people write have a model deciding what happens next. Today an LLM
> > > can
> > > > pick which downstream tasks run (@task.llm_branch), but like any branch
> > > it
> > > > can only select from tasks already wired in the Dag. It can't decide
> > the
> > > > order or the dependencies between them, so people either put the whole
> > > > agent loop inside one opaque task, losing per-step retries, logs and
> > > > approvals, or generate Dag files at parse time. A few shapes we keep
> > > seeing:
> > > >
> > > > - Incident triage: an agent reads an alert and picks diagnosis steps,
> > > > where which probe runs next depends on what the first one found, ending
> > > in
> > > > a fix or a page to a human.
> > > > - Support tickets: classify, then run some of validate, refund,
> > > > enrich, escalate and notify, in an order that depends on the ticket,
> > with
> > > > an approval task before anything that moves money.
> > > > - Data-quality failures: quarantine, backfill, re-validate and notify,
> > > > chosen per failure.
> > > > - Selective builds, with no LLM involved: an impact analysis decides
> > > > which models to rebuild and the edges between exactly those models.
> > > >
> > > > In each case the set of allowed actions is fixed and reviewed in the
> > Dag
> > > > file; only the selection and wiring change per run. Each action stays a
> > > > real task with its own retries, logs and pool, and the catalog plus a
> > > > validator keep the model inside what the author approved.
> > > >
> > > > On the name: neither of us likes "Dynamic Task Groups". It describes
> > the
> > > > mechanism, not what you get, which is building part of your Dag at run
> > > time.
> > > > We're keeping it for the AIP so the discussion doesn't get stuck on it,
> > > but
> > > > expect it to change before it ships.
> > > >
> > > > Two that came to my mind: "Plan Mode", as in a task group in plan mode,
> > > the
> > > > way coding agents plan before they act, and "Runtime
> > subgraphs/subdags",
> > > > which pairs with Task Loops and the @planner decorator. Suggestions
> > > welcome.
> > > >
> > > > What changed since the first draft:
> > > >
> > > > - Authoring is a @planner task inside a task group, plus a catalog of
> > > what
> > > > it
> > > >   may pick from (@triage.catalog, or catalog= / extra_catalog_items=).
> > > The
> > > >   TaskFlow and context-manager forms are side by side on the page and
> > are
> > > >   nearly identical. Catalog items can be operator classes or
> > import-path
> > > >   strings, so heavy operators don't have to be imported at parse time.
> > > >
> > > > - The plan uses the same schema as YAML Dags (tasks, uses, with,
> > needs),
> > > so
> > > >   Airflow has one vocabulary for declaring a graph of tasks. Templating
> > > is
> > > >   explicit, as in YAML Dags: a value is rendered only when marked with
> > > $t.
> > > >
> > > > - No subclassing. An LLM planner would itself be a planner, shipped by
> > a
> > > >   provider, with a validator on top. An @task.agent returning the plan
> > as
> > > >   structured output should also work with no extra machinery.
> > > >
> > > > - Selected entries are real task instances, not map indices of one
> > mapped
> > > >   task. The planner submits the plan through the Execution API while it
> > > is
> > > >   still running, and the API server creates the task instances, which
> > is
> > > the
> > > >   approach Task Loops took. The plan is stored against the region, so
> > > after
> > > > a
> > > >   clear and re-plan, the earlier plan and the task instances that ran
> > > under
> > > > it
> > > >   are still there to look at.
> > > >
> > > > - At parse time the group serializes as two nodes: the planner and a
> > > merge
> > > >   node. The merge is always an EmptyOperator, and anything downstream
> > of
> > > the
> > > >   group attaches to it.
> > > >
> > > > - Catalog membership is by exact class, so a subclass of an approved
> > > > operator
> > > >   is rejected. That can be loosened later; tightening it later would be
> > > much
> > > >   harder.
> > > >
> > > > Comments on the page or here are both fine.
> > > >
> > > > Thanks,
> > > > Kaxil
> > > >
> > > > On Tue, 18 Aug 2026 at 21:37, Przemysław Mirowski <[email protected]>
> > > wrote:
> > > >
> > > > > Thanks for the update Kaxil. Looks much better. I've added a couple
> > of
> > > new
> > > > > comments.
> > > > >
> > > > > On 2026/08/10 21:26:24 Kaxil Naik wrote:
> > > > > > Thanks for your feedback Przemysław, I updated the AIP
> > > > > >
> > > > > > On Fri, 7 Aug 2026 at 03:20, Przemysław Mirowski <[email protected]
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi Kaxil,
> > > > > > >
> > > > > > > I've added some comments to the AIP. In general, the idea looks
> > > > > > > interesting.
> > > > > > >
> > > > > > > Could we possibly rearrange the sections a bit? During the
> > reading,
> > > > > > > sometimes one thing was stated when the reader does not know a
> > > > > particular
> > > > > > > part of the change (e.g. merge policy - I added a comment next to
> > > it).
> > > > > > > Also, there are a lot of new definitions - maybe it would be good
> > > to
> > > > > create
> > > > > > > a section which will gather them all and shortly explain?
> > > > > > >
> > > > > > > On 2026/07/30 10:16:20 Kaxil Naik wrote:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I would like to start a discussion on AIP-113: Dynamic Task
> > > Groups.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
> > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440304823/AIP-113+Dynamic+Task+Groups
> > > > > > > >
> > > > > > > > tldr:
> > > > > > > > Dynamic Task Mapping made cardinality dynamic: run the same
> > task
> > > > > over a
> > > > > > > > collection discovered at run time. What is still static is the
> > > shape,
> > > > > > > > meaning which tasks
> > > > > > > > run in a region and how they depend on each other.
> > > > > > > >
> > > > > > > > Branching gets you half of that already, and I want to be
> > precise
> > > > > about
> > > > > > > > which half, because
> > > > > > > > I had this wrong in an earlier draft. `choose_branch` can
> > return
> > > a
> > > > > list,
> > > > > > > so
> > > > > > > > a branch already
> > > > > > > > selects an arbitrary subset of its declared successors. What a
> > > branch
> > > > > > > > cannot do is decide
> > > > > > > > the dependency edges *among* the tasks it selected. Selection
> > is
> > > > > solved;
> > > > > > > > ordering and wiring are not. That is the
> > > > > > > > gap this AIP is aimed at.
> > > > > > > >
> > > > > > > > The proposal is a Task Group whose members and internal edges
> > are
> > > > > decided
> > > > > > > > at run time from a fixed catalog the author declares. A
> > generator
> > > > > task
> > > > > > > > emits a plan, the group materializes
> > > > > > > > exactly the selected entries as real task instances, and the
> > > region
> > > > > > > > converges on a
> > > > > > > > synthesized merge node. The catalog is what serializes, so two
> > > runs
> > > > > of
> > > > > > > > different shapes share
> > > > > > > > one DAG version, and an entry the plan did not select never
> > > becomes a
> > > > > > > task
> > > > > > > > instance. The
> > > > > > > > planner can be a control table, an impact analysis, a triage
> > > rule,
> > > > > or an
> > > > > > > > LLM.
> > > > > > > >
> > > > > > > > ---
> > > > > > > >
> > > > > > > > Would love to hear your thoughts & get feedback. Please comment
> > > on
> > > > > the
> > > > > > > AIP
> > > > > > > > doc itself.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Kaxil
> > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: [email protected]
> > > > > > > For additional commands, e-mail: [email protected]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: [email protected]
> > > > > For additional commands, e-mail: [email protected]
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [email protected]
> > > For additional commands, e-mail: [email protected]
> > >
> > >
> >
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to