Lee-W commented on code in PR #66859:
URL: https://github.com/apache/airflow/pull/66859#discussion_r3264189169
##########
shared/state/src/airflow_shared/state/__init__.py:
##########
@@ -166,3 +173,41 @@ def cleanup(self) -> None:
retention policy. The backend is responsible for reading any relevant
config (e.g.
``[state_store] default_retention_days``) and deciding what to delete.
"""
+
+ def serialize_task_state_value(self, *, value: str, key: str, ti_id: str)
-> str:
+ """
+ Serialize a task state value before it is sent to the execution API
for db persistence.
+
+ Called by ``TaskStateAccessor.set()`` on the worker. The return value
is what gets
+ stored in the DB — typically a reference path (e.g. an S3 key) rather
than the
+ actual value. Default: return ``value`` unchanged.
+ """
+ return value
+
+ def deserialize_task_state_value(self, stored: str) -> str:
+ """
+ Resolve a stored task state string back to the actual value.
+
+ Called by ``TaskStateAccessor.get()`` after the stored string is
retrieved from
+ the execution API. Default: return ``stored`` unchanged.
+ """
+ return stored
+
+ def serialize_asset_state_value(self, *, value: str, key: str, asset_name:
str) -> str:
Review Comment:
> I think the naming is confusing you?
Somewhat, but not the main reason I pointed it out.
> For custom backends: there is a chance of orphaned external write if the
asset becomes inactive after the backend stores the value but before the
execution API confirms the reference. This is an edge case the custom backend
author needs to handle in their garbage cleanup imo.
sounds good. let's add a description somewhere in the doc so that users know
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
