Hi all, Thanks for writing the proposal. I've added one comment to the AIP regarding alternative solutions.
On 2026/07/31 08:12:47 Amogh Desai wrote: > Thanks for writing this up, Dheeraj and Shubham. > > I have some thoughts on simplifying it and have added it in the AIP > directly. > > Thanks & Regards, > Amogh Desai > > > On Fri, Jul 31, 2026 at 1:14 AM Constance Martineau via dev < > [email protected]> wrote: > > > Hi Djeeraj, > > > > I left a comment in the AIP, but it would be great if you could include > > some concrete examples of the workflow this serves. I understand why the > > workarounds are subpar and Airflow's limitations, but it's not obvious to > > me when I'd want to use this. A concrete workflow you run today would make > > the case much stronger. > > > > Thanks, > > Constance > > > > On Thu, Jul 30, 2026 at 2:28 PM Dheeraj Turaga <[email protected]> > > wrote: > > > > > Dear Airflow Community, > > > > > > Id like to start a discussion on AIP-115: On-Demand Task Sections > > > > > > > > > > > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440305022/AIP-115+On-Demand+Task+Sections > > > > > > TL;DR - AIP-115: On-Demand Task Sections > > > > > > Some Dags contain work that is expensive, slow, risky, or only > > > occasionally needed. Today an author can skip it (branching / > > > short-circuit) and lose any first-class way to run it afterward, > > > block the whole run on a human decision (HITL approval), or tell > > > users to clear the right tasks by hand. GitLab's manual pipeline > > > jobs are the closest existing analogue; Airflow has no Dag-level > > > equivalent. > > > > > > AIP-115 proposes a ManualGateOperator that lets an author mark a > > > section of a Dag as manual-only > > > > > > Scheduled runs skip past the gate and complete normally - nothing > > > waits on a human. The gated tasks are shown distinctly in the grid > > > and graph, and the gate offers a "Run manual section" action. > > > Triggering it runs that section for the specific Dag run you chose, > > > through the normal scheduler and executor path, with the usual logs, > > > retries, XComs, callbacks, and trigger-rule behavior. There's an > > > equivalent API endpoint for programmatic use. > > > > > > The result is an optional branch of a pipeline that's discoverable > > > from the Dag definition, visible in the UI, auditable, and runnable > > > per-run on demand - rather than a convention users have to know > > > about and reproduce by clearing tasks manually. The change is > > > additive: existing skipping, branching, and HITL behavior are > > > unchanged. > > > > > > The proposal covers the section-boundary semantics, task states, API > > > shape, and alternatives considered in detail > > > > > > ------ > > > Would love to hear your thoughts & feedback > > > > > > Regards, > > > Dheeraj > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
