subhash-0000 opened a new pull request, #60818:
URL: https://github.com/apache/airflow/pull/60818
## Description
This commit implements a new feature that allows users to run backfills
without unpausing the DAG, addressing issue #60768.
## Problem Statement
Currently, to perform a backfill, a DAG must be unpaused. However, for DAGs
that reprocess historical data but shouldn't run on their regular schedule
(e.g., when daily data is not available), this creates unnecessary executions.
Users need a way to run backfills while keeping the DAG paused to avoid useless
computation and logs.
## Solution
Added a new "Run backfill while keeping DAG paused" option that allows
backfills to execute even when the DAG is paused, without unpausing it. When
enabled, only the backfill runs execute—the DAG remains paused and won't be
triggered by its regular schedule.
## Changes
### Backend
- Added `keep_dag_paused` boolean column to `Backfill` model
- Modified scheduler logic in `get_queued_dag_runs_to_set_running()` and
`get_running_dag_runs_to_examine()` to allow backfill execution on paused DAGs
when `keep_dag_paused=True`
- Updated `BackfillPostBody` and `BackfillResponse` datamodels to include
the new field
- Modified `_create_backfill()` function to accept `keep_dag_paused`
parameter
- Updated `create_backfill` API route to pass `keep_dag_paused` parameter
### Frontend
- Added checkbox in `RunBackfillForm`: "Run backfill while keeping
{dag_name} paused"
- Added translation string for the new checkbox option
- Checkbox is disabled when "Unpause DAG on trigger" is selected (mutually
exclusive)
- Smart logic ensures `keep_dag_paused` is only sent when DAG is paused and
not being unpaused
### Database
- Created Alembic migration `0100_3_2_0_add_keep_dag_paused_to_backfill.py`
to add new column
### Tests
- Added unit tests for `Backfill` model with `keep_dag_paused` functionality
- Added API tests for backfill creation with `keep_dag_paused` parameter
- Added scheduler tests to verify backfills run on paused DAGs when enabled
- Updated existing tests to include `keep_dag_paused` in response assertions
### Documentation
- Added "Running backfills on paused DAGs" section to backfill documentation
- Updated UI instructions to explain both unpause options
- Documented use cases and when to use the feature
- Created newsfragment for release notes
## Backward Compatibility
✅ Fully backward compatible - `keep_dag_paused` defaults to `False`,
preserving existing behavior.
## Testing
All code changes are covered with tests:
- Model tests verify backfill creation and default values
- Scheduler tests verify paused DAGs can execute backfills
- API tests verify endpoint accepts and returns the parameter correctly
Closes: #60768
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: GitHub Copilot following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
