This is an automated email from the ASF dual-hosted git repository.
amoghrajesh pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new d878befd4c6 Document how retry policies and durable execution work
together (#73160)
d878befd4c6 is described below
commit d878befd4c6043c84f9d2f2e01cdf0c466b05895
Author: Amogh Desai <[email protected]>
AuthorDate: Fri Sep 18 16:04:00 2026 +0530
Document how retry policies and durable execution work together (#73160)
---
.../docs/core-concepts/resumable-tasks.rst | 39 ++++++++++++++++++++++
airflow-core/docs/core-concepts/tasks.rst | 8 +++++
providers/common/ai/docs/retry_policies.rst | 2 ++
3 files changed, 49 insertions(+)
diff --git a/airflow-core/docs/core-concepts/resumable-tasks.rst
b/airflow-core/docs/core-concepts/resumable-tasks.rst
index 163e11b3346..2fca68b978a 100644
--- a/airflow-core/docs/core-concepts/resumable-tasks.rst
+++ b/airflow-core/docs/core-concepts/resumable-tasks.rst
@@ -226,3 +226,42 @@ Comparison
- Airflow 2.2
- Airflow 3.3
- Airflow 3.2
+
+.. _concepts-resumable-tasks-retry-policies:
+
+Retry policies and durable execution
+------------------------------------
+
+Long-running tasks typically need two separate things together: a retry
+policy, and durable execution as defined above. Each is configurable per
+task, and together they let a workflow span days without losing progress.
+
+A **retry policy** decides whether a failed attempt gets another try, and
+how long to wait before it. This is :class:`~airflow.sdk.RetryPolicy` and
+:class:`~airflow.sdk.RetryDecision` (see :ref:`concepts:retry-policies`), or
+the LLM-driven
+:class:`~airflow.providers.common.ai.policies.retry.LLMRetryPolicy`, which
+uses a model to read the error and make that call. That policy adds its own
+``fallback_rules``, applied when the classification call itself fails, so
+the decision does not depend on the model being reachable (see
+:doc:`apache-airflow-providers-common-ai:retry_policies`).
+
+**Durable execution** is what that next attempt resumes from. Backed by the
+task state store described above, it is what lets a task recover a
+checkpoint written by the attempt before it, rather than starting over.
+
+Long-running tasks, or LLM-driven tasks like agentic workflows, benefit
+most from both: the retry policy decides whether the error is worth
+retrying at all, and the task state store is what the retry resumes from.
+
+Consider using both when a task:
+
+* Runs long enough that a worker crash mid-run is a real risk, and
+* Needs a retry decision more nuanced than "always retry" or "never
+ retry", for example retrying rate limits but failing outright on bad
+ credentials.
+
+The two operate independently. A task with ``retries=5`` and a checkpoint
+still stops for good after six failed attempts (the initial attempt plus five
+retries), but each of those retries picks up from the last checkpoint instead
of
+reprocessing files it already finished, or resubmitting a job that is still
running.
diff --git a/airflow-core/docs/core-concepts/tasks.rst
b/airflow-core/docs/core-concepts/tasks.rst
index 95d9846dd5e..2412dd6d5dc 100644
--- a/airflow-core/docs/core-concepts/tasks.rst
+++ b/airflow-core/docs/core-concepts/tasks.rst
@@ -314,6 +314,14 @@ DAG was triggered:
return RetryDecision.fail(reason="Backfill run -- not
retrying")
return RetryDecision.default()
+A retry policy only decides whether and when a task gets another attempt. It
+receives the full run ``context``, so it can inspect checkpointed progress
+through ``context["task_state_store"]`` when making that decision, but it
+never persists or restores that progress itself -- resuming from a checkpoint
+is the task's own work. For a task that also needs to resume from where it
+left off, pair a retry policy with the task state store described in
+:ref:`concepts-resumable-tasks-retry-policies`.
+
.. _concepts:task-instance-heartbeat-timeout:
Task Instance Heartbeat Timeout
diff --git a/providers/common/ai/docs/retry_policies.rst
b/providers/common/ai/docs/retry_policies.rst
index 94783907461..17c61e378c6 100644
--- a/providers/common/ai/docs/retry_policies.rst
+++ b/providers/common/ai/docs/retry_policies.rst
@@ -26,6 +26,8 @@ retry decisions. It works with any LLM provider supported by
pydantic-ai
(OpenAI, Anthropic, Bedrock, Vertex, Ollama, etc.).
For the core retry policy concepts, see
:doc:`apache-airflow:core-concepts/tasks`.
+If the task also needs to survive a worker crash without losing its progress,
+see :ref:`apache-airflow:concepts-resumable-tasks-retry-policies`.
Setup
-----
