This is an automated email from the ASF dual-hosted git repository.
kaxil pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 03c71e50eb2 Bump common.ai floor to pydantic-ai-slim>=1.71.0 and
document capabilities passthrough (#67444)
03c71e50eb2 is described below
commit 03c71e50eb23cbb501e79da7848d7a83347a237c
Author: Kaxil Naik <[email protected]>
AuthorDate: Sat May 30 01:48:11 2026 +0100
Bump common.ai floor to pydantic-ai-slim>=1.71.0 and document capabilities
passthrough (#67444)
1.71.0 is the first release that ships the `pydantic_ai.capabilities`
module (Thinking, WebSearch, ImageGeneration, MCP, etc.). Bumping the
floor lets users pass capabilities through `AgentOperator(agent_params=...)`
without operator-level support landing first.
The new example DAG demonstrates the pattern with Thinking, WebSearch,
and composition with SQLToolset. The agent.rst guide adds a short
"Capabilities (pydantic-ai)" section with a warning about the templated-
field serialization gap -- first-class `capabilities=` support is left
as a follow-up.
Generated metadata (README.rst, docs/index.rst, uv.lock) updated to match.
---
docs/spelling_wordlist.txt | 1 +
providers/common/ai/README.rst | 2 +-
providers/common/ai/docs/index.rst | 2 +-
providers/common/ai/docs/operators/agent.rst | 40 +++++++-
providers/common/ai/pyproject.toml | 2 +-
.../ai/example_dags/example_agent_capabilities.py | 114 +++++++++++++++++++++
uv.lock | 2 +-
7 files changed, 158 insertions(+), 5 deletions(-)
diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
index 222d69aab8a..fe93cebe41a 100644
--- a/docs/spelling_wordlist.txt
+++ b/docs/spelling_wordlist.txt
@@ -275,6 +275,7 @@ ColumnLineageDatasetFacet
CommandType
comparator
compat
+composable
Compute
compute
ComputeManagementClient
diff --git a/providers/common/ai/README.rst b/providers/common/ai/README.rst
index f9a587ec140..e4e21058ddb 100644
--- a/providers/common/ai/README.rst
+++ b/providers/common/ai/README.rst
@@ -56,7 +56,7 @@ PIP package Version required
``apache-airflow`` ``>=3.0.0``
``apache-airflow-providers-common-compat`` ``>=1.14.1``
``apache-airflow-providers-standard`` ``>=1.12.1``
-``pydantic-ai-slim`` ``>=1.34.0``
+``pydantic-ai-slim`` ``>=1.71.0``
========================================== ==================
Cross provider package dependencies
diff --git a/providers/common/ai/docs/index.rst
b/providers/common/ai/docs/index.rst
index 05e35ea22c8..adde6dbe9cc 100644
--- a/providers/common/ai/docs/index.rst
+++ b/providers/common/ai/docs/index.rst
@@ -108,7 +108,7 @@ PIP package Version required
``apache-airflow`` ``>=3.0.0``
``apache-airflow-providers-common-compat`` ``>=1.14.1``
``apache-airflow-providers-standard`` ``>=1.12.1``
-``pydantic-ai-slim`` ``>=1.34.0``
+``pydantic-ai-slim`` ``>=1.71.0``
========================================== ==================
Cross provider package dependencies
diff --git a/providers/common/ai/docs/operators/agent.rst
b/providers/common/ai/docs/operators/agent.rst
index 9f66b5aea3c..f1c13fb4490 100644
--- a/providers/common/ai/docs/operators/agent.rst
+++ b/providers/common/ai/docs/operators/agent.rst
@@ -255,6 +255,42 @@ skipped with a warning and will re-execute on retry
instead of replaying from
cache. The task itself still succeeds.
+.. _capabilities-passthrough:
+
+Capabilities (pydantic-ai)
+--------------------------
+
+pydantic-ai `capabilities <https://ai.pydantic.dev/capabilities/>`__ bundle
+tools, lifecycle hooks, instructions, and model settings into composable units.
+Common ones include ``Thinking`` (reasoning at a configurable effort level),
+``WebSearch``, ``WebFetch``, ``ImageGeneration``, and ``MCP``.
+
+``AgentOperator`` does not yet expose a first-class ``capabilities=`` kwarg,
+but anything passed through ``agent_params`` is forwarded to the underlying
+``Agent(...)`` constructor.
+
+.. exampleinclude::
/../../ai/src/airflow/providers/common/ai/example_dags/example_agent_capabilities.py
+ :language: python
+ :start-after: [START howto_operator_agent_capabilities_thinking]
+ :end-before: [END howto_operator_agent_capabilities_thinking]
+
+Capabilities compose with toolsets -- pydantic-ai merges tools from both.
+
+.. exampleinclude::
/../../ai/src/airflow/providers/common/ai/example_dags/example_agent_capabilities.py
+ :language: python
+ :start-after: [START howto_operator_agent_capabilities_composed]
+ :end-before: [END howto_operator_agent_capabilities_composed]
+
+.. warning::
+
+ ``agent_params`` is a templated field, which Airflow serializes by calling
+ ``str()`` on values it doesn't natively understand. Capability instances
+ are not yet round-trip-safe through DAG serialization, so the examples
+ below construct them inside the ``@dag`` function -- not at module level.
+ First-class ``capabilities=`` support on ``AgentOperator`` (with proper
+ serializer hooks) is tracked as a follow-up.
+
+
Parameters
----------
@@ -273,7 +309,9 @@ Parameters
:class:`~airflow.providers.common.ai.toolsets.logging.LoggingToolset` so that
every tool call is logged in real time. Default ``True``.
- ``agent_params``: Additional keyword arguments passed to the pydantic-ai
- ``Agent`` constructor (e.g. ``retries``, ``model_settings``).
+ ``Agent`` constructor (e.g. ``retries``, ``model_settings``,
``capabilities``).
+ See :ref:`capabilities-passthrough` for how to enable pydantic-ai
capabilities
+ such as ``Thinking``, ``WebSearch``, and ``ImageGeneration``.
- ``usage_limits``: Optional pydantic-ai ``UsageLimits`` enforced on every
agent run (initial run, durable replay, and HITL regeneration). Use it to
cap requests, tokens, or tool calls per task -- agents are particularly
diff --git a/providers/common/ai/pyproject.toml
b/providers/common/ai/pyproject.toml
index 8a0e6bacbe1..0f684c26941 100644
--- a/providers/common/ai/pyproject.toml
+++ b/providers/common/ai/pyproject.toml
@@ -69,7 +69,7 @@ dependencies = [
"apache-airflow>=3.0.0",
"apache-airflow-providers-common-compat>=1.14.1",
"apache-airflow-providers-standard>=1.12.1",
- "pydantic-ai-slim>=1.34.0",
+ "pydantic-ai-slim>=1.71.0",
]
# The optional dependencies should be modified in place in the generated file
diff --git
a/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent_capabilities.py
b/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent_capabilities.py
new file mode 100644
index 00000000000..3d069acdf33
--- /dev/null
+++
b/providers/common/ai/src/airflow/providers/common/ai/example_dags/example_agent_capabilities.py
@@ -0,0 +1,114 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Example DAGs demonstrating pydantic-ai capabilities via ``agent_params``.
+
+Capabilities (https://ai.pydantic.dev/capabilities/) are pydantic-ai's
+composable units for thinking, web search, image generation, MCP, and more.
+``AgentOperator`` forwards anything in ``agent_params`` to the underlying
+``Agent(...)`` constructor, so capabilities work today without operator-level
+support. A first-class ``capabilities=`` kwarg is on the roadmap.
+"""
+
+from __future__ import annotations
+
+from pydantic_ai.capabilities import Thinking, WebSearch
+
+from airflow.providers.common.ai.operators.agent import AgentOperator
+from airflow.providers.common.ai.toolsets.sql import SQLToolset
+from airflow.providers.common.compat.sdk import dag
+
+# ---------------------------------------------------------------------------
+# 1. Thinking capability: enable model reasoning at a configurable effort level
+# ---------------------------------------------------------------------------
+
+
+# [START howto_operator_agent_capabilities_thinking]
+@dag(tags=["example"])
+def example_agent_capabilities_thinking():
+ AgentOperator(
+ task_id="reasoner",
+ prompt="Walk through the steps to compute the 10th Fibonacci number,
then give the answer.",
+ llm_conn_id="pydanticai_default",
+ system_prompt="You are a careful mathematician. Think before
answering.",
+ agent_params={
+ "capabilities": [Thinking(effort="high")],
+ },
+ )
+
+
+# [END howto_operator_agent_capabilities_thinking]
+
+example_agent_capabilities_thinking()
+
+
+# ---------------------------------------------------------------------------
+# 2. WebSearch capability: provider-adaptive web search as a single declaration
+# ---------------------------------------------------------------------------
+
+
+# [START howto_operator_agent_capabilities_web_search]
+@dag(tags=["example"])
+def example_agent_capabilities_web_search():
+ AgentOperator(
+ task_id="researcher",
+ prompt="Summarize the latest Apache Airflow 3.x release notes from
airflow.apache.org.",
+ llm_conn_id="pydanticai_default",
+ system_prompt="You are a release-notes summarizer. Cite the source
URL.",
+ agent_params={
+ "capabilities": [WebSearch()],
+ },
+ )
+
+
+# [END howto_operator_agent_capabilities_web_search]
+
+example_agent_capabilities_web_search()
+
+
+# ---------------------------------------------------------------------------
+# 3. Composing capabilities with toolsets
+# ---------------------------------------------------------------------------
+# Capabilities and toolsets compose: pydantic-ai merges tools from both.
+
+
+# [START howto_operator_agent_capabilities_composed]
+@dag(tags=["example"])
+def example_agent_capabilities_composed():
+ AgentOperator(
+ task_id="analyst",
+ prompt="Cross-reference our top customers with their recent public
news. Think first.",
+ llm_conn_id="pydanticai_default",
+ system_prompt=(
+ "You are a sales analyst. Query the database for customers, then
search the web "
+ "for recent news. Reason carefully about which leads to surface."
+ ),
+ toolsets=[
+ SQLToolset(
+ db_conn_id="postgres_default",
+ allowed_tables=["customers", "orders"],
+ max_rows=20,
+ ),
+ ],
+ agent_params={
+ "capabilities": [Thinking(effort="medium"), WebSearch()],
+ },
+ )
+
+
+# [END howto_operator_agent_capabilities_composed]
+
+example_agent_capabilities_composed()
diff --git a/uv.lock b/uv.lock
index caf92c6b934..d80641fe5ca 100644
--- a/uv.lock
+++ b/uv.lock
@@ -4326,7 +4326,7 @@ requires-dist = [
{ name = "llama-index-llms-openai", marker = "extra == 'llamaindex'",
specifier = ">=0.6.0" },
{ name = "pyarrow", marker = "python_full_version >= '3.14' and extra ==
'parquet'", specifier = ">=22.0.0" },
{ name = "pyarrow", marker = "python_full_version < '3.14' and extra ==
'parquet'", specifier = ">=18.0.0" },
- { name = "pydantic-ai-slim", specifier = ">=1.34.0" },
+ { name = "pydantic-ai-slim", specifier = ">=1.71.0" },
{ name = "pydantic-ai-slim", extras = ["anthropic"], marker = "extra ==
'anthropic'" },
{ name = "pydantic-ai-slim", extras = ["bedrock"], marker = "extra ==
'bedrock'" },
{ name = "pydantic-ai-slim", extras = ["google"], marker = "extra ==
'google'" },
