kaxil commented on code in PR #74729:
URL: https://github.com/apache/airflow/pull/74729#discussion_r4339910692
##########
providers/common/ai/docs/structured_output.rst:
##########
@@ -50,6 +50,34 @@ loaded Dag's tasks at startup and registers each declared
class, so it also work
for mapped producers (``.expand(...)``) and for workers that load Dags from a
cache that bypasses operator construction.
+The worker finds no class to register when ``output_type`` is a list of output
+types such as ``[Summary, int]``, an output marker such as
+``ToolOutput(Summary)``, an output function, a ``TypeAliasType`` or a
+``NewType``. For these output types, ``AgentOperator`` and ``@task.agent`` push
+each model in the output as a JSON-compatible dict, including models inside a
+list, tuple or dict. A model is pushed as a dict even if its class is listed in
+``[core] allowed_deserialization_classes``, or if another task in the same Dag
+declares the class as its ``output_type``. A downstream task therefore reads
+``result["field"]``, not ``result.field``.
+
+In a JSON-compatible dict, values such as ``datetime``, ``date``, ``UUID`` and
+``Decimal`` arrive as strings, tuples and sets arrive as lists, and ``Enum``
+members arrive as their values. In HITL review, the released common.ai 1.0.0
+dumped a top-level model of these output types with plain ``model_dump()``
+after the reviewer approved the model. On Airflow 3.3 and later without
+``serialize_output=True``, the ``datetime`` fields of such a model therefore
+arrive as strings after an upgrade, not as ``datetime`` objects.
Review Comment:
`providers-common-ai/1.1.0rc1` still has the plain `model_dump()` on this
path, so this should say 1.0.0 and 1.1.0. The non-HITL change isn't covered
either: on 3.3+ these output types pushed the instance before, so someone who
allow-listed the class (or declared it as another task's `output_type`) got
`result.field` working and now gets a dict and an `AttributeError`. Since both
change the type a downstream task receives, could this also get a `.. note::`
at the top of `changelog.rst`, like the 1.0.0 ones?
##########
providers/common/ai/src/airflow/providers/common/ai/operators/agent.py:
##########
@@ -1120,6 +1129,23 @@ def _complete_run(self, context: Context, result: Any,
*, attempt_usage: RunUsag
self._clear_usage_budget(context)
return output
+ def _convert_output_for_xcom(self, output: Any) -> Any:
+ """Return the XCom value, with dicts in place of models that a
downstream task could not load."""
+ if self._serialize_model_output and isinstance(output, BaseModel):
+ return output.model_dump()
+ if not _CORE_WALKER:
+ # iter_pydantic_models could not be imported, so the worker
registers no class.
+ # _serialize_model_output is always True when that import fails,
so a model at the top level was
+ # already dumped above. The class of a model inside a list, tuple
or dict is not registered
+ # either, so this call passes an empty set of registered classes
to dump that model too.
+ return _dump_unregistered_models(output, set())
+ # A downstream task can load a model from XCom only if the worker
registered the model's class.
+ # The worker registers the classes that iter_pydantic_models finds in
output_type.
+ # iter_pydantic_models finds no class in a list of output types, an
output marker, an output
+ # function, a TypeAliasType or a NewType. For these output types the
operator pushes each model
+ # as a dict, including models inside a list, tuple or dict.
+ return _dump_unregistered_models(output,
set(iter_pydantic_models(self.output_type)))
Review Comment:
With `do_xcom_push=False` nothing gets pushed, but this still runs
`model_dump(mode="json")`, which raises for a model with non-UTF-8 `bytes`
(`UnicodeDecodeError`) or an `arbitrary_types_allowed` field
(`PydanticSerializationError`). Such a task with `ToolOutput(...)` on 3.3+
succeeded before and now fails after the model calls are paid for. Returning
`output` early when `not self.do_xcom_push` would avoid it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
