kaxil commented on code in PR #74694:
URL: https://github.com/apache/airflow/pull/74694#discussion_r4268546945
##########
providers/common/ai/docs/changelog.rst:
##########
@@ -25,6 +25,12 @@
Changelog
---------
+.. note::
+ ``LoggingToolset`` is no longer part of the public API and is no longer
exported from
Review Comment:
The `airflow.providers.common.ai.toolsets` re-export came from #74449 and
never shipped (0.11.0rc1's `toolsets/__init__.py` doesn't have it). The path
users actually have is
`airflow.providers.common.ai.toolsets.logging.LoggingToolset`, which is in
0.10.0 and now raises `ImportError`. Could the note name that path and give a
replacement for the removed "use it directly with any pydantic-ai `Agent`"
example, e.g. `Agent(..., capabilities=[ToolLoggingCapability()])`?
##########
providers/common/ai/docs/stability.rst:
##########
@@ -99,6 +102,110 @@ Pydantic AI toolsets, but the Pydantic AI class they
inherit from can change.
timeout under the default ``on_approval_timeout="fail"``, fails the
task, except on
``@task.llm_branch``, where it skips the downstream tasks unless
``fail_on_reject=True``.
+ * - ``decision_policy`` on ``@task.llm`` and ``@task.llm_branch``, and
+ ``min_confidence`` on
+ :class:`~airflow.providers.common.ai.policies.decision.BranchOption`
+ - Without a ``min_confidence`` the operator behaves as it does without a
policy. With
+ one, it returns (or branches on) the answer itself only when the least
confident
+ field that reported a confidence is at or above the bar. Under the bar,
+ ``on_uncertain="review"`` (the default; needs Airflow 3.1+) opens the
same human
+ review as ``require_approval``, and ``"fail"`` fails the task with
+ ``LowConfidenceError``. A model that reports no confidence counts as
under the bar.
+ ``require_approval=True`` always asks a person. On ``@task.llm_branch``
a
+ ``BranchOption.min_confidence`` replaces the policy's bar for that
branch, and with
+ ``allow_multiple_branches=True`` the strictest bar among the picked
branches applies.
+ The operator pushes a ``decision`` XCom recording the confidence and
the bar that
+ applied. ``@task.llm_sql``, ``@task.llm_file_analysis`` and
+ ``@task.llm_schema_compare`` reject a policy with a bar when they are
created.
+ * - Per-tool approval on ``AgentOperator`` (``tool_approval_timeout``,
+ ``on_tool_approval_timeout`` and ``tool_approval_assigned_users``)
+ - Needs Airflow 3.3 or later; on earlier versions a tool marked for
approval fails the
+ task. A call to a tool marked with Pydantic AI's approval API pauses
the task before
+ the tool runs and asks on the Required Actions page, showing the tool
name and its
+ arguments with secrets masked. **Approve** runs the call; **Reject**
tells the agent
+ the call was denied, with the reviewer's reason, and the run carries
on. A task
+ instance asks at most once per Dag run, retries and clears included; a
second
Review Comment:
Main keeps per-tool approval experimental because of this sentence. The only
reason `_pause_for_tool_approval` gives for the guard is that a repeat request
keeps the first one's subject and body. Since #73235 (in 3.4.0b1) a repeat
request replaces them, and #74688 refuses an answer to the replaced request, so
lifting the once-per-run limit is on the way. Promising
`ToolApprovalAlreadyRequestedError` as stable would make that a breaking
change. I'd keep this row experimental, or at least name the once-per-run limit
as not covered.
##########
providers/common/ai/docs/stability.rst:
##########
@@ -99,6 +102,110 @@ Pydantic AI toolsets, but the Pydantic AI class they
inherit from can change.
timeout under the default ``on_approval_timeout="fail"``, fails the
task, except on
``@task.llm_branch``, where it skips the downstream tasks unless
``fail_on_reject=True``.
+ * - ``decision_policy`` on ``@task.llm`` and ``@task.llm_branch``, and
+ ``min_confidence`` on
+ :class:`~airflow.providers.common.ai.policies.decision.BranchOption`
+ - Without a ``min_confidence`` the operator behaves as it does without a
policy. With
+ one, it returns (or branches on) the answer itself only when the least
confident
+ field that reported a confidence is at or above the bar. Under the bar,
+ ``on_uncertain="review"`` (the default; needs Airflow 3.1+) opens the
same human
+ review as ``require_approval``, and ``"fail"`` fails the task with
+ ``LowConfidenceError``. A model that reports no confidence counts as
under the bar.
+ ``require_approval=True`` always asks a person. On ``@task.llm_branch``
a
+ ``BranchOption.min_confidence`` replaces the policy's bar for that
branch, and with
+ ``allow_multiple_branches=True`` the strictest bar among the picked
branches applies.
+ The operator pushes a ``decision`` XCom recording the confidence and
the bar that
+ applied. ``@task.llm_sql``, ``@task.llm_file_analysis`` and
+ ``@task.llm_schema_compare`` reject a policy with a bar when they are
created.
+ * - Per-tool approval on ``AgentOperator`` (``tool_approval_timeout``,
+ ``on_tool_approval_timeout`` and ``tool_approval_assigned_users``)
+ - Needs Airflow 3.3 or later; on earlier versions a tool marked for
approval fails the
+ task. A call to a tool marked with Pydantic AI's approval API pauses
the task before
+ the tool runs and asks on the Required Actions page, showing the tool
name and its
+ arguments with secrets masked. **Approve** runs the call; **Reject**
tells the agent
+ the call was denied, with the reviewer's reason, and the run carries
on. A task
+ instance asks at most once per Dag run, retries and clears included; a
second
+ request fails the task with ``ToolApprovalAlreadyRequestedError``.
+ ``tool_approval_timeout`` (default ``None``, wait for as long as it
takes) bounds the
+ pause. ``on_tool_approval_timeout`` is ``"fail"`` (default) or
``"deny"``, which needs
+ a timeout; a timeout never approves. ``tool_approval_assigned_users``
limits who may
+ decide. Beside ``durable=True``, ``enable_hitl_review=True``, a
``CodeMode``
+ capability or a ``SandboxToolset`` that provisions its own sandbox, a
tool marked for
+ approval fails the task, except one called from code mode's
``run_code``, which does
+ not run and is reported to the model.
+ * -
:class:`~airflow.providers.common.ai.toolsets.object_storage.ObjectStorageToolset`
+ - Exposes ``list_files``, ``get_file_info`` and ``read_file``, each named
+ ``<tool_prefix>_<name>`` when ``tool_prefix`` is set, rooted at
``path`` and read-only.
+ A path that is absolute, carries a scheme or climbs out of the root is
refused. A
+ ``list_files`` result holds at most ``max_files`` entries.
``read_file`` returns a
+ text file a window of lines at a time, at most ``max_output_bytes``
bytes, with the
+ offset to continue from, and a Parquet or Avro file as its row count,
schema and first
+ rows. It refuses images, PDFs and other binary files, and any file
larger than
+ ``max_read_bytes``.
+ * -
:class:`~airflow.providers.common.ai.managed_agents.base.BaseManagedAgentHook`,
+
:class:`~airflow.providers.common.ai.toolsets.managed_agent.ManagedAgentToolset`
and
+
:class:`~airflow.providers.common.ai.toolsets.managed_agent.BaseManagedAgentToolset`
+ - A vendor hook that adopts ``BaseManagedAgentHook`` implements
``resolve_agent``,
+ ``get_agent_capabilities`` and ``invoke_agent``, and
``hook.agent(name)`` binds one
+ agent; the Amazon ``BedrockAgentCoreHook`` and the Google
``AgentEngineHook`` do so.
+ ``ManagedAgentToolset`` exposes a managed agent as one tool named
``tool_name`` that
+ takes a single ``prompt`` and gives the model the answer's text. A
rejection the model
+ can fix by rephrasing is returned to it, up to ``max_retries`` times
(default ``1``);
+ a terminal or transient error fails the call and is left to the task's
retry
+ settings. With ``durable=True`` a completed call is replayed from the
cache only when
+ ``replayable=True``. A subclass of ``BaseManagedAgentToolset``
implements
+ ``agent_ref`` and ``invoke_sync`` (or ``invoke``).
+ * - ``@task.llm_sql`` and
+
:class:`~airflow.providers.common.ai.operators.llm_sql.LLMSQLQueryOperator`
+ - Returns the SQL the model generates from the prompt and never runs it.
Schema
+ context is ``schema_context`` when given, otherwise the tables in
``table_names``
+ introspected through ``db_conn_id``. With ``validate_sql=True`` (the
default) the
+ task fails unless the result is a single statement of a type in
``allowed_sql_types``
+ (by default ``SELECT``, ``UNION``, ``INTERSECT`` and ``EXCEPT``). A
reviewer's edit
+ under ``require_approval`` is validated again against
``allowed_sql_types``, whether
+ or not ``validate_sql`` is set. ``datasource_config`` is not covered:
it reads object
+ storage through DataFusion and is experimental; see below.
+ * - ``@task.llm_file_analysis`` and
+
:class:`~airflow.providers.common.ai.operators.llm_file_analysis.LLMFileAnalysisOperator`
+ - Reads ``file_path``, a file or a prefix on object storage or a local
path, and sends
+ its content to the model in one call. At most ``max_files`` files from
a prefix are
+ included and the rest are noted in the prompt; the task fails when a
file exceeds
+ ``max_file_size_bytes`` or the files together exceed
``max_total_size_bytes``.
+ Text sent to the model is cut to ``max_text_chars``, and CSV, Parquet
and Avro inputs
+ are sampled to ``sample_rows`` rows. PNG, JPG and PDF files are
attached only with
+ ``multi_modal=True``. The output follows the rules of ``@task.llm``.
+ * - ``@task.llm_schema_compare`` and
+
:class:`~airflow.providers.common.ai.operators.llm_schema_compare.LLMSchemaCompareOperator`
+ - Compares at least two schemas, from ``table_names`` on each of
``db_conn_ids`` or from
+ ``data_sources``, and returns a ``SchemaCompareResult`` as a dict. It
takes no
+ ``output_type``. ``context_strategy="basic"`` sends column names and
types;
+ ``"full"`` (the default) adds primary keys, foreign keys and indexes. A
+ ``system_prompt`` replaces the default schema-comparison guidance, not
the fixed
+ preamble and the schemas that surround it. ``data_sources`` is not
covered: it reads a
+ source whose connection is not a ``DbApiHook`` through DataFusion and
is experimental;
+ see below.
+ * - :class:`~airflow.providers.common.ai.hooks.langchain.LangChainHook` and
+
:func:`~airflow.providers.common.ai.toolsets.langchain_bridge.airflow_toolset_to_langchain_tools`
+ - ``get_chat_model()`` and ``get_embedding_model()`` return LangChain
models for
+ identifiers in ``provider:name`` form, from ``llm_model`` or the
connection's
+ ``model`` and ``embed_model`` extras. The connection's password is
passed as
+ ``api_key`` and its host as ``base_url``. ``embed_conn_id`` falls back
to
+ ``llm_conn_id``. The bridge returns one LangChain ``StructuredTool``
per tool of the
+ toolset, with the tool's argument schema, and masks secrets in what a
tool returns.
+ An argument that fails validation, a ``ModelRetry`` (up to the tool's
+ ``max_retries``) or a ``ToolFailed`` reaches the model as an error
result. Once the
+ retries are used up, and for any other exception, the call raises and
the run fails;
+ the exception class, ``ToolCallError``, is in the experimental
``tools`` package.
+ * - Tool-call logging: ``enable_tool_logging`` on ``AgentOperator`` and
+
:class:`~airflow.providers.common.ai.toolsets.logging.ToolLoggingCapability`
+ - With ``enable_tool_logging=True`` (the default), each call to a tool
supplied through
+ ``toolsets=``, ``agent_params["tools"]`` or a capability logs its name
and duration at
+ INFO inside a ``::group::`` block and its arguments at DEBUG. A call
the run pauses
+ for approval, or that asks for a retry, deferral or skip, is logged at
INFO and
+ re-raised; any other exception is logged at ERROR with a traceback and
re-raised.
+ Output tools, provider-native tools and tools another capability adds
through its own
+ wrapper are not logged. When ``capabilities`` already holds a
+ ``ToolLoggingCapability``, the operator adds none and its ``logger`` is
used.
Experimental features
Review Comment:
This conflicts with #74679, which merged to main this morning (`git
merge-tree` against main conflicts here and in `toolsets/index.rst`), and the
two PRs classify six features the opposite way. This PR makes per-tool
approval, `DecisionPolicy` / `BranchOption.min_confidence`, managed agent
toolsets and the LangChain hook and bridge stable, while main keeps them
experimental. Main has `@task.llm_batch` and the framework-neutral `tools`
interface stable, while this PR lists them as experimental.
`frameworks/index.rst` also merges cleanly into "the framework-neutral tool
interface under them is stable", which would contradict the `tools` row here.
Can you rebase so we can settle those six rows before going further?
##########
providers/common/ai/docs/frameworks/index.rst:
##########
@@ -103,9 +103,12 @@ Tested versions
- Pins OpenTelemetry at 1.42.1 or lower, below the version in Airflow's
constraints
file. Airflow itself accepts 1.42.1.
-These are the versions the adapter tests have been run against. CI does not
run those
-tests: its environment has ``mcp`` 2.2 and OpenTelemetry 1.44, which Strands
and ADK
-exclude. They run in an environment with the framework installed.
+These are the versions the adapter tests have been run against. The main test
matrix
+skips those tests: its environment has ``mcp`` 2.2 and OpenTelemetry 1.44,
which Strands
+and ADK exclude. A separate CI job installs one framework at a time and runs
them, on
+amd64 only, on the ``main`` branch only, and only when ``common.ai`` Python
source or
Review Comment:
Not quite "only when": `run_agent_framework_tests` goes through
`_should_be_run`, which also returns `True` whenever selective checks decide
full tests are needed (canary and push runs, the `full tests needed` label,
environment file changes). The same sentence is in `stability.rst` around line
280.
##########
providers/common/ai/src/airflow/providers/common/ai/operators/agent.py:
##########
@@ -277,7 +277,7 @@ class AgentOperator(CancellableAgentRunMixin, BaseOperator,
HITLReviewMixin, Usa
in the serialized Dag, and cannot be combined with this argument (the
task fails when it runs).
:param enable_tool_logging: When ``True`` (default), wraps the agent's
- assembled function toolset in a ``LoggingToolset`` that logs tool calls
+ assembled function toolset in a ``ToolLoggingCapability`` that logs
tool calls
Review Comment:
The toolset isn't wrapped in a `ToolLoggingCapability`. The operator appends
the capability, and its `get_wrapper_toolset` does the wrapping. Maybe "adds a
`ToolLoggingCapability`, which wraps the agent's assembled function toolset",
plus the rule from the stable row that a `ToolLoggingCapability` already in
`capabilities` is used instead of adding a second one.
##########
providers/common/ai/docs/decision_models.rst:
##########
@@ -20,8 +20,9 @@ Decision models
.. note::
- Experimental: this can change or be removed in a minor release of this
provider.
- See :ref:`howto/stability`.
+ Experimental: using a decision model with ``ClassifierRetryPolicy`` can
change or be
Review Comment:
`ClassifierRetryPolicy` is experimental as a whole (its class docstring
still says so), not only when used with a decision model. It also has its own
`min_confidence`, so "`min_confidence` ... are stable" reads as covering it.
Maybe "`ClassifierRetryPolicy` is experimental. `DecisionPolicy` and
`BranchOption.min_confidence` are stable."
##########
providers/common/ai/docs/stability.rst:
##########
@@ -99,6 +102,110 @@ Pydantic AI toolsets, but the Pydantic AI class they
inherit from can change.
timeout under the default ``on_approval_timeout="fail"``, fails the
task, except on
``@task.llm_branch``, where it skips the downstream tasks unless
``fail_on_reject=True``.
+ * - ``decision_policy`` on ``@task.llm`` and ``@task.llm_branch``, and
+ ``min_confidence`` on
+ :class:`~airflow.providers.common.ai.policies.decision.BranchOption`
+ - Without a ``min_confidence`` the operator behaves as it does without a
policy. With
+ one, it returns (or branches on) the answer itself only when the least
confident
+ field that reported a confidence is at or above the bar. Under the bar,
+ ``on_uncertain="review"`` (the default; needs Airflow 3.1+) opens the
same human
+ review as ``require_approval``, and ``"fail"`` fails the task with
+ ``LowConfidenceError``. A model that reports no confidence counts as
under the bar.
+ ``require_approval=True`` always asks a person. On ``@task.llm_branch``
a
+ ``BranchOption.min_confidence`` replaces the policy's bar for that
branch, and with
+ ``allow_multiple_branches=True`` the strictest bar among the picked
branches applies.
+ The operator pushes a ``decision`` XCom recording the confidence and
the bar that
+ applied. ``@task.llm_sql``, ``@task.llm_file_analysis`` and
+ ``@task.llm_schema_compare`` reject a policy with a bar when they are
created.
+ * - Per-tool approval on ``AgentOperator`` (``tool_approval_timeout``,
+ ``on_tool_approval_timeout`` and ``tool_approval_assigned_users``)
+ - Needs Airflow 3.3 or later; on earlier versions a tool marked for
approval fails the
+ task. A call to a tool marked with Pydantic AI's approval API pauses
the task before
+ the tool runs and asks on the Required Actions page, showing the tool
name and its
+ arguments with secrets masked. **Approve** runs the call; **Reject**
tells the agent
+ the call was denied, with the reviewer's reason, and the run carries
on. A task
+ instance asks at most once per Dag run, retries and clears included; a
second
+ request fails the task with ``ToolApprovalAlreadyRequestedError``.
+ ``tool_approval_timeout`` (default ``None``, wait for as long as it
takes) bounds the
+ pause. ``on_tool_approval_timeout`` is ``"fail"`` (default) or
``"deny"``, which needs
+ a timeout; a timeout never approves. ``tool_approval_assigned_users``
limits who may
+ decide. Beside ``durable=True``, ``enable_hitl_review=True``, a
``CodeMode``
+ capability or a ``SandboxToolset`` that provisions its own sandbox, a
tool marked for
+ approval fails the task, except one called from code mode's
``run_code``, which does
+ not run and is reported to the model.
+ * -
:class:`~airflow.providers.common.ai.toolsets.object_storage.ObjectStorageToolset`
+ - Exposes ``list_files``, ``get_file_info`` and ``read_file``, each named
+ ``<tool_prefix>_<name>`` when ``tool_prefix`` is set, rooted at
``path`` and read-only.
+ A path that is absolute, carries a scheme or climbs out of the root is
refused. A
+ ``list_files`` result holds at most ``max_files`` entries.
``read_file`` returns a
+ text file a window of lines at a time, at most ``max_output_bytes``
bytes, with the
+ offset to continue from, and a Parquet or Avro file as its row count,
schema and first
+ rows. It refuses images, PDFs and other binary files, and any file
larger than
+ ``max_read_bytes``.
+ * -
:class:`~airflow.providers.common.ai.managed_agents.base.BaseManagedAgentHook`,
+
:class:`~airflow.providers.common.ai.toolsets.managed_agent.ManagedAgentToolset`
and
+
:class:`~airflow.providers.common.ai.toolsets.managed_agent.BaseManagedAgentToolset`
+ - A vendor hook that adopts ``BaseManagedAgentHook`` implements
``resolve_agent``,
+ ``get_agent_capabilities`` and ``invoke_agent``, and
``hook.agent(name)`` binds one
+ agent; the Amazon ``BedrockAgentCoreHook`` and the Google
``AgentEngineHook`` do so.
+ ``ManagedAgentToolset`` exposes a managed agent as one tool named
``tool_name`` that
+ takes a single ``prompt`` and gives the model the answer's text. A
rejection the model
+ can fix by rephrasing is returned to it, up to ``max_retries`` times
(default ``1``);
+ a terminal or transient error fails the call and is left to the task's
retry
+ settings. With ``durable=True`` a completed call is replayed from the
cache only when
+ ``replayable=True``. A subclass of ``BaseManagedAgentToolset``
implements
+ ``agent_ref`` and ``invoke_sync`` (or ``invoke``).
+ * - ``@task.llm_sql`` and
+
:class:`~airflow.providers.common.ai.operators.llm_sql.LLMSQLQueryOperator`
+ - Returns the SQL the model generates from the prompt and never runs it.
Schema
+ context is ``schema_context`` when given, otherwise the tables in
``table_names``
+ introspected through ``db_conn_id``. With ``validate_sql=True`` (the
default) the
+ task fails unless the result is a single statement of a type in
``allowed_sql_types``
+ (by default ``SELECT``, ``UNION``, ``INTERSECT`` and ``EXCEPT``). A
reviewer's edit
+ under ``require_approval`` is validated again against
``allowed_sql_types``, whether
+ or not ``validate_sql`` is set. ``datasource_config`` is not covered:
it reads object
+ storage through DataFusion and is experimental; see below.
+ * - ``@task.llm_file_analysis`` and
+
:class:`~airflow.providers.common.ai.operators.llm_file_analysis.LLMFileAnalysisOperator`
+ - Reads ``file_path``, a file or a prefix on object storage or a local
path, and sends
+ its content to the model in one call. At most ``max_files`` files from
a prefix are
+ included and the rest are noted in the prompt; the task fails when a
file exceeds
+ ``max_file_size_bytes`` or the files together exceed
``max_total_size_bytes``.
+ Text sent to the model is cut to ``max_text_chars``, and CSV, Parquet
and Avro inputs
+ are sampled to ``sample_rows`` rows. PNG, JPG and PDF files are
attached only with
+ ``multi_modal=True``. The output follows the rules of ``@task.llm``.
+ * - ``@task.llm_schema_compare`` and
+
:class:`~airflow.providers.common.ai.operators.llm_schema_compare.LLMSchemaCompareOperator`
+ - Compares at least two schemas, from ``table_names`` on each of
``db_conn_ids`` or from
+ ``data_sources``, and returns a ``SchemaCompareResult`` as a dict. It
takes no
+ ``output_type``. ``context_strategy="basic"`` sends column names and
types;
+ ``"full"`` (the default) adds primary keys, foreign keys and indexes. A
+ ``system_prompt`` replaces the default schema-comparison guidance, not
the fixed
+ preamble and the schemas that surround it. ``data_sources`` is not
covered: it reads a
+ source whose connection is not a ``DbApiHook`` through DataFusion and
is experimental;
+ see below.
+ * - :class:`~airflow.providers.common.ai.hooks.langchain.LangChainHook` and
+
:func:`~airflow.providers.common.ai.toolsets.langchain_bridge.airflow_toolset_to_langchain_tools`
+ - ``get_chat_model()`` and ``get_embedding_model()`` return LangChain
models for
+ identifiers in ``provider:name`` form, from ``llm_model`` or the
connection's
+ ``model`` and ``embed_model`` extras. The connection's password is
passed as
+ ``api_key`` and its host as ``base_url``. ``embed_conn_id`` falls back
to
+ ``llm_conn_id``. The bridge returns one LangChain ``StructuredTool``
per tool of the
+ toolset, with the tool's argument schema, and masks secrets in what a
tool returns.
+ An argument that fails validation, a ``ModelRetry`` (up to the tool's
+ ``max_retries``) or a ``ToolFailed`` reaches the model as an error
result. Once the
+ retries are used up, and for any other exception, the call raises and
the run fails;
+ the exception class, ``ToolCallError``, is in the experimental
``tools`` package.
+ * - Tool-call logging: ``enable_tool_logging`` on ``AgentOperator`` and
+
:class:`~airflow.providers.common.ai.toolsets.logging.ToolLoggingCapability`
+ - With ``enable_tool_logging=True`` (the default), each call to a tool
supplied through
+ ``toolsets=``, ``agent_params["tools"]`` or a capability logs its name
and duration at
+ INFO inside a ``::group::`` block and its arguments at DEBUG. A call
the run pauses
+ for approval, or that asks for a retry, deferral or skip, is logged at
INFO and
+ re-raised; any other exception is logged at ERROR with a traceback and
re-raised.
Review Comment:
This puts `ToolFailed` under "any other exception is logged at ERROR", but
`_LoggingToolset.call_tool` logs it at INFO alongside `ModelRetry`, and
`ObjectStorageToolset` raises it for every refused path. `toolsets/logging.rst`
already says "report a failed tool result"; could this row list it too?
##########
providers/common/ai/docs/stability.rst:
##########
@@ -113,77 +220,69 @@ Everything this provider ships that is not in the table
above is experimental.
- Why it is experimental
* -
:class:`~airflow.providers.common.ai.policies.retry.ClassifierRetryPolicy`
(:doc:`decision_models`)
- - The confidence threshold, the fallback order and the behaviour when the
classifier
- is unavailable are still settling.
- * - :class:`~airflow.providers.common.ai.policies.decision.DecisionPolicy`,
and
- ``min_confidence`` on
- :class:`~airflow.providers.common.ai.policies.decision.BranchOption`
- (:doc:`approval_gates`)
- - New. The threshold semantics and the recorded decision may change as
they are
- used.
+ - It reads a decision model, and the ``typesafe:`` and ``system-one:``
prefixes need
+ ``pydantic-ai-slim`` 2.45 and 2.53, newer than the 2.33 floor of the
provider's
+ other extras.
* - ``@task.llm_batch`` and batch adapters (:doc:`operators/llm_batch`)
- - Submission, re-attachment on retry, cancellation and handling of
partial results
- are still settling, and ``BatchAdapter`` has no implementation outside
this
- provider yet.
+ - ``BatchAdapter`` has two implementations, OpenAI and Anthropic, both in
this
+ provider and none outside it.
* - ``durable=True`` on ``AgentOperator`` (:doc:`durable_execution`)
- - Which tool results are replayed on retry will change, so that a tool
can declare
- whether its result may be reused.
- * - Per-tool approval on ``AgentOperator`` (``tool_approval_timeout``,
- ``on_tool_approval_timeout`` and ``tool_approval_assigned_users``;
- :doc:`tool_approval`)
- - New, and needs Airflow 3.3. How a paused run resumes may change.
- * - Code mode (:doc:`code_mode`), the Agent Skills toolset
- (:doc:`toolsets/skills`) and the ``shields`` extra (used in
:doc:`capabilities`)
- - Thin integrations of packages outside this provider whose APIs are still
- changing: ``pydantic-ai-harness``, ``pydantic-ai-skills`` and
- ``pydantic-ai-shields``.
+ - Only ``ManagedAgentToolset`` can declare, through ``replayable``, that
its results
Review Comment:
It isn't only `ManagedAgentToolset`: `caching_toolset.py` reads `replayable`
with `getattr` from the innermost toolset of any tool, and
`BaseManagedAgentToolset` defaults it to `False`. Main's reason for this row
("which tool results are replayed on retry will change") avoids the problem, so
keeping it when resolving the conflict may be simplest.
##########
providers/common/ai/docs/stability.rst:
##########
@@ -113,77 +220,69 @@ Everything this provider ships that is not in the table
above is experimental.
- Why it is experimental
* -
:class:`~airflow.providers.common.ai.policies.retry.ClassifierRetryPolicy`
(:doc:`decision_models`)
- - The confidence threshold, the fallback order and the behaviour when the
classifier
- is unavailable are still settling.
- * - :class:`~airflow.providers.common.ai.policies.decision.DecisionPolicy`,
and
- ``min_confidence`` on
- :class:`~airflow.providers.common.ai.policies.decision.BranchOption`
- (:doc:`approval_gates`)
- - New. The threshold semantics and the recorded decision may change as
they are
- used.
+ - It reads a decision model, and the ``typesafe:`` and ``system-one:``
prefixes need
+ ``pydantic-ai-slim`` 2.45 and 2.53, newer than the 2.33 floor of the
provider's
+ other extras.
* - ``@task.llm_batch`` and batch adapters (:doc:`operators/llm_batch`)
- - Submission, re-attachment on retry, cancellation and handling of
partial results
- are still settling, and ``BatchAdapter`` has no implementation outside
this
- provider yet.
+ - ``BatchAdapter`` has two implementations, OpenAI and Anthropic, both in
this
+ provider and none outside it.
* - ``durable=True`` on ``AgentOperator`` (:doc:`durable_execution`)
- - Which tool results are replayed on retry will change, so that a tool
can declare
- whether its result may be reused.
- * - Per-tool approval on ``AgentOperator`` (``tool_approval_timeout``,
- ``on_tool_approval_timeout`` and ``tool_approval_assigned_users``;
- :doc:`tool_approval`)
- - New, and needs Airflow 3.3. How a paused run resumes may change.
- * - Code mode (:doc:`code_mode`), the Agent Skills toolset
- (:doc:`toolsets/skills`) and the ``shields`` extra (used in
:doc:`capabilities`)
- - Thin integrations of packages outside this provider whose APIs are still
- changing: ``pydantic-ai-harness``, ``pydantic-ai-skills`` and
- ``pydantic-ai-shields``.
+ - Only ``ManagedAgentToolset`` can declare, through ``replayable``, that
its results
+ must not be replayed. For every other tool a cached result is replayed
on retry, and
+ a tool has no way to declare that its result must not be reused.
+ * - Code mode (:doc:`code_mode`)
+ - It depends on ``pydantic-ai-harness``, which is pre-1.0; the
``code-mode`` extra asks
+ for 0.24.0 or later.
+ * - The Agent Skills toolset (:doc:`toolsets/skills`)
+ - The ``skills`` extra is built on ``pydantic-ai-skills``. Native
progressive
+ disclosure is tracked upstream in
+ `pydantic/pydantic-ai#5230
<https://github.com/pydantic/pydantic-ai/issues/5230>`__,
+ and the extra is to be revisited once that lands.
+ * - The ``shields`` extra (used in :doc:`capabilities`)
+ - It depends on ``pydantic-ai-shields``, which is pre-1.0 (the extra asks
for 0.3.4 or
+ later), and this provider has no test of it.
* - :class:`~airflow.providers.common.ai.toolsets.sandbox.SandboxToolset`
and its
backends (:doc:`sandbox/index`)
- - The sandbox runtime belongs to the backend; ownership and cleanup
across worker
- failures are still being designed.
- * - LangChain and LlamaIndex hooks and the LangChain tool bridge
- (:doc:`hooks/langchain`, :doc:`toolsets/langchain`,
:doc:`hooks/llamaindex`)
- - Each follows the API of a framework that changes often.
+ - A worker killed outright skips teardown. The ``sbx`` backend has no
server-side
+ lifetime, so its microVM and workspace survive and nothing reclaims them
+ (:doc:`sandbox/backends`).
+ * - The LlamaIndex hook (:doc:`hooks/llamaindex`)
+ - It depends on ``llama-index-core``, which is pre-1.0; the
``llamaindex`` extra asks
+ for 0.14.5 or later.
* - The retrieval operators: ``DocumentLoaderOperator``,
``LlamaIndexEmbeddingOperator`` and ``LlamaIndexRetrievalOperator``
(:doc:`rag_pipelines`)
- - New. The shape of the pipeline, from loading documents through
embedding to
- retrieval, has not settled.
+ - The two LlamaIndex operators depend on ``llama-index-core``, which is
pre-1.0.
+ ``DocumentLoaderOperator`` needs no AI framework; whether it belongs in
this provider's
+ stable surface is an open decision, not a technical reason.
* -
:class:`~airflow.providers.common.ai.toolsets.datafusion.DataFusionToolset`
(:doc:`toolsets/datafusion`)
- Runs on the DataFusion engine of ``common.sql``, which pins
``datafusion`` below 52
- and follows its API.
- * - Managed agent toolsets (:doc:`toolsets/managed_agent`)
- - A contract for vendor providers that no vendor implements yet.
+ because the DataFusion 52 crate is not supported with ``iceberg-core``.
+ * - ``datasource_config`` on ``@task.llm_sql`` and ``data_sources`` on
+ ``@task.llm_schema_compare`` (:doc:`operators/llm_sql`,
+ :doc:`operators/llm_schema_compare`)
+ - They read object storage, or any source whose connection is not a
``DbApiHook``,
+ through the DataFusion engine of ``common.sql``, which pins
+ ``datafusion`` below 52 because the DataFusion 52 crate is not
supported with
+ ``iceberg-core``.
* - OpenTelemetry spans (``[common.ai] otel_export_enabled`` and
``capture_content``; :doc:`observability`)
- - Span names and attributes come from Pydantic AI's instrumentation and
the
- OpenTelemetry GenAI conventions, which are still in development.
- * - :class:`~airflow.providers.common.ai.toolsets.logging.LoggingToolset`
used directly
- (:doc:`toolsets/logging`)
- - The wrapper ``AgentOperator`` applies for ``enable_tool_logging``; its
constructor
- may change.
- * - ``@task.llm_schema_compare``, ``@task.llm_file_analysis`` and
``@task.llm_sql``
- (:doc:`operators/llm_schema_compare`,
:doc:`operators/llm_file_analysis`,
- :doc:`operators/llm_sql`)
- - Each adds its own input handling to ``@task.llm``: schema
introspection, file
- sampling, or SQL validation. Their options are still settling.
+ - Span names and attributes come from Pydantic AI's instrumentation,
which follows the
+ OpenTelemetry GenAI semantic conventions. This provider only turns it
on, so it does
Review Comment:
The provider does control part of this: `observability.py` pins
`_SEMCONV_VERSION = 5` so a change in pydantic-ai's default doesn't shift the
span format between provider releases. The reason that still holds is main's,
that the GenAI semantic conventions are still in development.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
