kaxil commented on code in PR #76861:
URL: https://github.com/apache/airflow/pull/76861#discussion_r4537980208
##########
providers/common/ai/src/airflow/providers/common/ai/durable/journal.py:
##########
@@ -160,6 +177,28 @@ def record(self, payload: Any) -> bool:
self.durable_run.journal.stats.recorded[self.kind] += 1
log.debug("Durable: recorded step", position=self.position,
step=self.name)
return True
+ self._not_recorded()
+ return False
+
+ def cannot_record(self, error: Exception) -> None:
+ """
+ Note that the step's result has no form the journal can hold, such as
a value with no JSON form.
+
+ :raises DurableJournalError: When the journal fails on such results,
as it does by default.
+ Otherwise the step is left unrecorded, as when storage skips a
write, and runs again on retry.
+ """
+ if not self.replayable:
+ # Never recorded anyway, so there is nothing to fail or report.
+ return
+ if self.durable_run.journal.fail_on_unrecordable:
+ # The same value fails the same way on every retry, so retrying
the task cannot help.
+ raise DurableJournalError(
+ f"durable execution could not record a step's result, because
it is not JSON-serializable: {error}"
+ ) from error
+ log.warning("Durable: step result is not JSON-serializable",
step=self.name, error=str(error))
+ self._not_recorded()
+
+ def _not_recorded(self) -> None:
Review Comment:
Done in 16f516fafde.
##########
providers/common/ai/docs/durable_execution.rst:
##########
@@ -366,11 +366,15 @@ A toolset can opt out of replay. ``ManagedAgentToolset``
does by default
(``replayable=False``), because a managed agent acts on a system Airflow
cannot observe:
its calls run again on every attempt, while the steps around them still replay.
-A result has to survive a JSON round trip to be recorded. One that cannot be
encoded
-fails the task with ``DurableJournalError`` and no retry, since it would fail
the same
-way every time; the model could not have read it either. A write the task
state store
-refuses does not fail the task: the task log names
-the step, and a retry runs it again.
+A result has to survive a JSON round trip to be recorded. Bytes a tool
returns, on their
Review Comment:
Added in 16f516fafde. What happens to the unconverted ones depends on the
pydantic-ai version (from 2.66 it records dataclass bytes itself as base64), so
the docs say that instead of promising one outcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
