Agreed that a timeout override shouldn't need a Lookup. AIP-117 doesn't require
one for overrides either: Lookups are only for authors who want a value to
depend on a param, the run type or a Variable, and the overrides field works on
any Dag as is.
To keep one interface, what if the PR puts the timeout under an "overrides"
object, e.g. {"overrides": {"dag_run_timeout": "PT30M"}}? AIP-117 would later
add "tasks" and "pools" keys to the same object, so users get one place for
per-run changes to what the Dag defines. Rules can differ per key: the timeout
needs only trigger permission and works on every surface your PR covers, while
the AIP-117 keys need Dag edit and admin-nominated pools or queues. Storage
doesn't need to change - I think a dedicated column, as in your PR, is better
for something the scheduler checks every loop.
Later, a Lookup could let an author set dagrun_timeout per run type (e.g.
longer for backfills) without anyone passing it at trigger time. An explicit
value in the request would still win over the Lookup.
Either way, thank you for taking the time to review AIP-117. Looking forward to
your thoughts and more discussion about it. If there are no major concerns, I'd
like to move it to a vote soon.
Thanks,
Ramit
> On Nov 3, 2026, at 6:55 AM, Hussein Awala <[email protected]> wrote:
>
> I’m already reviewing AIP-117, and I like the proposal. Its focus is on
> task scheduling parameters, whereas this proposal controls the timeout of
> the Dag run itself.
> By changing Dag code, I mean requiring the author to explicitly define a
> lookup to make the timeout configurable. IMHO, this should be a native
> option when creating a Dag run, with `dagrun_timeout` as the default and an
> optional override for that individual run, without requiring any lookup
> declaration or custom param handling.
>
>> On Tue, Nov 3, 2026 at 2:36 PM Elad Kalif via dev <[email protected]>
>> wrote:
>>
>> If I get it right when you create a new run you wish to submit a value that
>> is different from the one in the code file?
>> It has some similarities with AIP-117 that lays the ground for pools and
>> queues. I think dagrun timeout is a simpler case because it doesn't involve
>> the scheduler loop.
>> My point is that we should have the same user interface like the one
>> suggested in AIP-117
>>
>> https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/446071748/AIP-117+Per-Run+Task+Scheduling+Controls
>>
>>> On Tue, Nov 3, 2026 at 8:53 AM Jarek Potiuk <[email protected]> wrote:
>>>
>>> No problem. It's a nice "operational" improvement that makes a lot of
>>> sense..
>>>
>>>> On Mon, Nov 2, 2026 at 12:13 PM Hussein Awala <[email protected]> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> While working on a custom backfill DAG to trigger another DAG today, I
>>> was
>>>> surprised to find that there is no way to override the DAG-level
>>>> dagrun_timeout for an individual run.
>>>>
>>>> The DAG-level timeout works well for regular scheduled runs. However,
>>>> manual runs and custom backfills can process larger batches of data and
>>>> need more time. I’d like to set a different timeout for those runs
>>> without
>>>> changing the DAG definition or affecting other runs.
>>>>
>>>> I opened PR #76311 <https://github.com/apache/airflow/pull/76311> to
>>>> support this. The idea is straightforward: persist an optional timeout
>>>> override on each DAG run and have the scheduler use it when set,
>> falling
>>>> back to the DAG-level timeout otherwise. Omitting the override
>> preserves
>>>> the existing behavior.
>>>>
>>>> The diff is fairly large because the parameter needs to be supported
>>> across
>>>> the APIs, clients, UI, backfills, and SDK, along with tests and
>> generated
>>>> schemas. The core change is a nullable database column and the
>> scheduler
>>>> logic to select the effective timeout. Support in TriggerDagRunOperator
>>> is
>>>> planned as a separate follow-up.
>>>>
>>>> I’m sharing this here for visibility and to hear any objections or
>>> concerns
>>>> I may have missed. Does anyone see a problem with allowing this
>> override,
>>>> or an edge case we should consider?
>>>>
>>>> Thanks,
>>>> Hussein
>>>>
>>>
>>
