amoghrajesh opened a new issue, #72929:
URL: https://github.com/apache/airflow/issues/72929
### Body
#72100 added discard-by-default behavior for `task_state_store` entries when
a task instance
is cleared via the `POST /dags/{dag_id}/clearTaskInstances` endpoint —
checkpoints are
discarded unless `keep_task_state` is set, so a cleared task starts over
instead of silently
resuming from a stale checkpoint after the underlying code/data was fixed.
That fix only applies to that one endpoint. Every other way of clearing a
task instance still
keeps task state unconditionally, because each of them calls
`clear_task_instances()` directly
instead of going through the new discard logic:
- Clearing an entire Dag run (the "Clear Run" dialog / API) —
`perform_clear_dag_run` → `dag.clear()` → `clear_task_instances()`.
- Marking a task as failed/success, which clears downstream tasks as a side
effect.
- `airflow dags clear` / `airflow tasks clear` (core CLI) —
`cli/commands/dag_command.py` → `clear_task_instances()` directly.
### Follow-up work
Thread `keep_task_state` (or an equivalent opt-out) through
`clear_task_instances()` itself, so
every caller (REST endpoint, Dag-run clear, CLI, mark-as-success/failed)
converges on the same
discard-by-default behavior. This likely needs:
- A `keep_task_state` parameter on `clear_task_instances()`.
- Plumbing it through `dag.clear()`, `perform_clear_dag_run`, and the core
CLI's
`airflow dags clear` / `airflow tasks clear`.
- Deciding the right default/UX for callers that have no request body to
carry the flag today
(e.g. CLI defaults, mark-as-success/failed).
### Acceptance criteria
- Clearing a Dag run discards `task_state_store` entries by default,
matching the task-instance
clear endpoint, with an equivalent opt-out to keep state.
- `airflow dags clear` / `airflow tasks clear` behave consistently with the
REST endpoints.
- Docs (`resumable-tasks.rst`) no longer need to caveat which clear surface
actually discards.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
