Hi, So I have been through the AIP as well as this thread. Overall, I am in favour of this AIP because:
1. There is value in being able to intervene without redesigning or redeploying a DAG. Dedicated pools, asset dependencies and human-in-the-loop tasks can address anticipated scenarios, but they require the DAG to have been designed accordingly. An on-call engineer may need to prevent one task from starting during an incident without stopping unrelated work or waiting for a deployment. David raised a few alternatives that effectively achieve what this task/ti hold feature does but I don’t think any are as ergonomic. 2. I see Ash raised some objections on the basis that this may violate the ‘pipeline as code’ philosophy. But I don’t think that conflict is inherent. Don’t we already allow users to intervene through the UI and API, such as pausing DAGs, clearing tasks and marking task states. A task-instance hold is no different i.e. the code defines the workflow, while the hold temporarily controls whether a particular task instance can start. I think the biggest sticking point (already raised by others) is the potential performance implications. This of course depends on the final agreed scope (task/tis or just tis) but it would be interesting to see what the actual planned metrics for the benchmark would be. I have left a comment on the confluence page. Thanks, Sameer. On Thu, 5 Nov 2026 at 00:55, <[email protected]> wrote: > Hi all, > > Elad and I would like to start the discussion on AIP-128, which > proposes introducing task holds in Airflow. > > Users sometimes need one task to wait while unrelated work > continues—for example, while correcting bad data or waiting for a > vendor API to recover. A hold lets them prevent that task from > starting without pausing the whole Dag. > > AIP: > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/462815942/AIP-128+Task+TaskInstance+Hold > > Demo (1.5 min): > > https://cwiki.apache.org/confluence/download/attachments/462815942/task-holds-demo-short.mp4?api=v2 > > The proposal supports two kinds of hold: > > - Task instance hold: one task in one run, for example publish_report > in today's run while bad data is fixed. > - Task hold: one task in every run, for example while a vendor API is > unavailable. > > The scheduler checks holds before picking tasks to start. An attempt > it has already picked may still run. A hold doesn't change the task's > state; other state changes, such as upstream failure, happen as usual. > Once all holds end, normal scheduling can continue. > > Every hold has a reason, an optional end time and an audit log entry. > We have a working prototype, shown in the video. > > The AIP explains the key design decisions and their trade-offs. We'd > particularly welcome feedback on whether a hold should only block a > task's start, and whether the two proposed scopes are sufficient for a > first version. > > Please comment on the AIP doc, or reply here. > > Thanks, > Shivam > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
