jason810496 commented on code in PR #71356: URL: https://github.com/apache/airflow/pull/71356#discussion_r3756859564
########## airflow-core/docs/core-concepts/resumable-tasks.rst: ########## @@ -126,6 +126,27 @@ existing job on retry instead of submitting a new one. For more details and a working example, see :class:`~airflow.sdk.ResumableJobMixin`. +**Clearing a task is treated the same as a retry** + +Clearing a task instance does not delete its ``task_state_store`` rows -- they are only removed +when the ``dag_run`` itself is deleted, or by :ref:`airflow state-store clean +<task-and-asset-state-store-cleanup>`. For a checkpointed task this is usually what you want: +clearing resumes from the last checkpoint rather than starting over. + +For an operator with durable execution, it means clearing a task whose external job already +succeeded reads that stored result back and returns immediately, without resubmitting the job. If +you want clearing to always resubmit regardless of a prior success, set +``[state_store] clear_on_success = True``, which deletes a task's state store rows automatically +when it moves to ``SUCCESS`` (see :doc:`/administration-and-deployment/task-and-asset-state-store`). + +This does not guarantee the external job is still there to reconnect to, though. Clearing a task +that is actively running (``deferrable=False``) stops the worker process, which runs the Review Comment: Small nit, no strong opnion. ```suggestion that is actively running with ``deferrable=False`` stops the worker process, which runs the ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
