dheerajturaga opened a new pull request, #73763:
URL: https://github.com/apache/airflow/pull/73763
A paused Dag never starts the runs you trigger or backfill, so the only way
to run them today is to unpause the Dag, which also resumes its schedule. That
doesn't work for Dags that only reprocess historical data on demand: the
scheduled runs fail or waste resources. An earlier attempt (#60818) let runs
execute on a paused Dag and was closed, because a paused Dag should never run.
This PR builds on the new `draining` state (#72407) instead. Draining
already means "create no scheduled runs, finish the existing ones, then pause".
Manual runs, backfills and asset materializations can now start that drain
themselves. The run executes, the scheduler creates nothing new, and the Dag
lands back on paused once its runs finish. The paused state keeps its meaning,
and no new scheduling state is added.
closes: #60768
### What changes for users
- **REST API**: `POST /dags/{dag_id}/dagRuns`, `POST /backfills` and `POST
/assets/{asset_id}/materialize` accept `drain_dag` (default `false`). The Dag
starts draining in the same transaction that creates the run or backfill, so a
rejected request leaves the Dag untouched.
- **Permissions**: because draining changes the Dag's scheduling state,
`drain_dag` also requires the permission `PATCH /dags/{dag_id}` needs to pause
the Dag (`PUT` on the Dag), on top of the permission to create the run.
- **UI**: on a paused Dag, the trigger, backfill and materialize forms
replace the "Unpause on trigger" checkbox with a collapsible "Dag is paused"
section. It offers three choices: *Unpause on trigger* (default, same as
today), *Run without resuming the schedule* (drains), or *Leave paused*. The
collapsed header shows the current choice. When draining is chosen and the Dag
already has unfinished runs, a warning says they will run too.
- **airflowctl**: `dags trigger` and `backfill create` get `--drain-dag`
through the regenerated datamodels.
- **Docs**: the drain section in `core-concepts/dags.rst` and the backfill
UI steps are updated, and the drain release note (`72407.significant.rst`)
gains a paragraph, since both ship in 3.4.0.
### Behaviour worth reviewing
- Draining lets **every** unfinished run proceed. On a paused Dag, runs that
were already sitting in the queue start as well. Starting only the new run
would need a per-run exemption from the paused check, which is the approach
#60818 was closed for. The UI warns about this instead.
- On an active Dag, `drain_dag` works like draining from the pause toggle:
the schedule stops now and the Dag ends paused.
- `drain_dag` is nullable on the trigger body because `airflowctl dags
trigger` sends unset fields as `null`. A strict boolean would reject every
plain `airflowctl dags trigger`.
- `DagModel.start_drain` locks the Dag row before the run is inserted.
Otherwise the drain finalizer could pause an already-draining Dag at the same
moment, leaving the new run stuck in the queue. Taking the lock before the
insert also avoids upgrading the foreign-key lock the insert takes on MySQL.
### Testing
Run locally:
- **Breeze pytest (SQLite)**:
- Trigger, materialize and backfill route tests, `test_backfill.py`, and
the backfill/drain tests in `test_scheduler_job.py`: 302 passed.
- After the permission commit, the drain and permission route tests again:
12 passed.
- I confirmed three kinds of new test fail without their change: the
stale-state test for `start_drain`, the three permission tests, and the UI test
for a stale unfinished-run count.
- **airflowctl**: 399 passed.
- **UI (vitest)**: the full suite passes, except `src/queryClient.test.ts`.
That file isn't touched here; it failed intermittently under the full parallel
run and passes on its own.
- **Static checks**: `prek` pre-commit and manual stages, and
`mypy-airflow-core`, pass.
- **Docs**: `breeze build-docs apache-airflow` with spellcheck passed,
before two small wording edits to the RST.
- **Not completed locally**: the full parallel core suites that selective
checks request, and the provider suites. These are left to CI.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 5.5)
Generated-by: Claude Code (Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
