amoghrajesh commented on code in PR #76021:
URL: https://github.com/apache/airflow/pull/76021#discussion_r4493000324
##########
airflow-core/docs/core-concepts/task-state-store.rst:
##########
@@ -316,4 +316,18 @@ When ``[state_store] clear_on_success = True``, all task
state store keys for a
``clear_on_success`` clears **task state store only**. Asset store is
scoped to the asset, not the task instance, and is never affected by this
setting. Asset store persists across runs and must be cleared explicitly.
+.. note::
+
+ With a custom ``[workers] state_store_backend``, the worker deletes the
backend's entries only
+ after the success has been reported, once the task's callbacks and
listeners have run. If that
+ deletion fails the task stays successful and the failure is logged with the
Dag id, run id and
+ task id.
+
+ Those entries are then orphaned: the API server has already removed the
metadata-DB references,
+ ``airflow state-store clean`` covers the metastore backend only, and
+ :meth:`~airflow.sdk.state.BaseStoreBackend.cleanup` is a no-op unless your
backend implements
+ it. Because the storage scope includes the run id, the next Dag run writes
elsewhere rather than
+ overwriting them. Removing them is up to your storage's own lifecycle or
TTL policy, or a
+ ``cleanup()`` implementation on your backend.
Review Comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
