Hi Djeeraj, I left a comment in the AIP, but it would be great if you could include some concrete examples of the workflow this serves. I understand why the workarounds are subpar and Airflow's limitations, but it's not obvious to me when I'd want to use this. A concrete workflow you run today would make the case much stronger.
Thanks, Constance On Thu, Jul 30, 2026 at 2:28 PM Dheeraj Turaga <[email protected]> wrote: > Dear Airflow Community, > > Id like to start a discussion on AIP-115: On-Demand Task Sections > > > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440305022/AIP-115+On-Demand+Task+Sections > > TL;DR - AIP-115: On-Demand Task Sections > > Some Dags contain work that is expensive, slow, risky, or only > occasionally needed. Today an author can skip it (branching / > short-circuit) and lose any first-class way to run it afterward, > block the whole run on a human decision (HITL approval), or tell > users to clear the right tasks by hand. GitLab's manual pipeline > jobs are the closest existing analogue; Airflow has no Dag-level > equivalent. > > AIP-115 proposes a ManualGateOperator that lets an author mark a > section of a Dag as manual-only > > Scheduled runs skip past the gate and complete normally - nothing > waits on a human. The gated tasks are shown distinctly in the grid > and graph, and the gate offers a "Run manual section" action. > Triggering it runs that section for the specific Dag run you chose, > through the normal scheduler and executor path, with the usual logs, > retries, XComs, callbacks, and trigger-rule behavior. There's an > equivalent API endpoint for programmatic use. > > The result is an optional branch of a pipeline that's discoverable > from the Dag definition, visible in the UI, auditable, and runnable > per-run on demand - rather than a convention users have to know > about and reproduce by clearing tasks manually. The change is > additive: existing skipping, branching, and HITL behavior are > unchanged. > > The proposal covers the section-boundary semantics, task states, API > shape, and alternatives considered in detail > > ------ > Would love to hear your thoughts & feedback > > Regards, > Dheeraj >
