This is an automated email from the ASF dual-hosted git repository. kaxil pushed a commit to branch commonai-capability-map in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 2d43ebdb61cd8410c045159201f7e96a4dc75ddf Author: Kaxil Naik <[email protected]> AuthorDate: Wed Oct 14 12:13:10 2026 +0100 Add a feature map to the Common AI docs Turn features.rst into a feature map: one row per feature with the page that documents it, the extra it needs, the oldest Airflow version it runs on, and whether it is stable. Link it from the landing page next to Why Common AI, and add labels for the three sections it links into. --- providers/common/ai/docs/approval_gates.rst | 2 + providers/common/ai/docs/features.rst | 404 ++++++++++++++++++++++++-- providers/common/ai/docs/frameworks/index.rst | 2 + providers/common/ai/docs/index.rst | 4 +- providers/common/ai/docs/observability.rst | 4 +- 5 files changed, 386 insertions(+), 30 deletions(-) diff --git a/providers/common/ai/docs/approval_gates.rst b/providers/common/ai/docs/approval_gates.rst index c40eee8e2e9..a42ccf2786a 100644 --- a/providers/common/ai/docs/approval_gates.rst +++ b/providers/common/ai/docs/approval_gates.rst @@ -82,6 +82,8 @@ user row id as a string, not the username. This needs Airflow 3.1+. On Airflow clearing the task re-runs it against the existing review row, so a changed list does not take effect. +.. _approval-gates:uncertain: + Reviewing uncertain output -------------------------- diff --git a/providers/common/ai/docs/features.rst b/providers/common/ai/docs/features.rst index 3885569eced..7d11bc0122e 100644 --- a/providers/common/ai/docs/features.rst +++ b/providers/common/ai/docs/features.rst @@ -17,34 +17,382 @@ .. _howto/features: -LLM and agent features -====================== - -These settings change how a model call or an agent run behaves, independent of which operator -you use. Each is a parameter on the operator or decorator. - -- :doc:`structured_output`: ``output_type`` returns a typed Pydantic object through XCom - instead of a string. -- :doc:`message_history`: ``message_history`` carries a conversation across agent runs. -- :doc:`memory`: pydantic-ai-harness ``Memory`` gives an agent notes it writes in one Dag - run and reads in the next. -- :doc:`capabilities`: ``capabilities=`` adds pydantic-ai capabilities such as ``Thinking`` and - ``WebSearch``, and ``pydantic-ai-shields`` guardrails, to an agent. -- :doc:`code_mode`: the ``CodeMode`` capability lets the model call several tools from one - Python snippet instead of one round trip per call. -- :doc:`approval_gates`: ``require_approval=True`` pauses an LLM operator until a person - approves, edits or rejects the output. -- :doc:`hitl_review`: ``enable_hitl_review=True`` opens an iterative review loop on an agent, - with a chat UI and REST API for the reviewer. -- :doc:`tool_approval`: a tool marked with pydantic-ai's approval API pauses the agent before - the call runs, until a person approves or rejects it. - -The last three are different tools for different jobs: an approval gate is a one-shot decision on -one output, a HITL review is a conversation with a running agent, and a tool approval is a -decision on one action before it happens. - -Making retries cheap with ``durable=True`` is a reliability feature and lives under -:doc:`operations`. +Feature map +=========== + +Most of the provider's operators and decorators run +`Pydantic AI <https://ai.pydantic.dev/>`__ agents, with the model taken from an Airflow +connection. A feature that calls a model through those connections also needs the extra +for your model vendor (:doc:`model_providers`); the **Extra** column lists what it needs +on top of that, and a bare name there is an extra of this provider. +**Airflow** is the oldest Airflow version the feature runs on; the provider itself needs +2.11. A **stable** feature keeps its public parameters and their defaults across minor +releases from 1.0.0 on, along with the behaviour listed for it in :doc:`stability`. An +**experimental** one can change or be removed in a minor release. **Third-party** marks a +package from outside this provider, whose own stability applies. + +Run a model or an agent +----------------------- + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`@task.llm <operators/llm>` + - Sends one prompt and returns the answer as text or a typed object. + - None + - 2.11 + - Stable + * - :doc:`@task.agent <operators/agent>` + - Runs an agent that calls tools until it has an answer. + - None + - 2.11 + - Stable + * - :doc:`@task.llm_branch <operators/llm_branch>` + - Lets the model pick which downstream tasks run. + - None + - 2.11 + - Stable + * - :doc:`@task.llm_sql <operators/llm_sql>` + - Writes a SQL query from a question and checks it, without running it. + - ``sql``; for files, ``apache-airflow-providers-common-sql[datafusion]`` + - 2.11 + - Stable + * - :doc:`@task.llm_file_analysis <operators/llm_file_analysis>` + - Sends files from object storage, including images and PDFs, to the model with a + prompt. + - ``parquet`` or ``avro`` for those formats + - 2.11 + - Stable + * - :doc:`@task.llm_schema_compare <operators/llm_schema_compare>` + - Compares table schemas across databases and files and grades each mismatch. + - ``common.sql``; for files, ``apache-airflow-providers-common-sql[datafusion]`` + - 2.11 + - Stable + * - :doc:`@task.llm_batch <operators/llm_batch>` + - Sends many prompts through the OpenAI or Anthropic batch API at about half the + per-token price. + - None + - 2.11 + - Stable + * - :doc:`Document and RAG pipelines <rag_pipelines>` + - Loads documents, embeds them and retrieves the passages closest to a question. + - ``llamaindex``, and install the vendor's package for vendors other than OpenAI; + ``pdf`` or ``docx`` for those formats + - 2.11 + - Experimental + +Choose a model +-------------- + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Models and providers <model_providers>` + - Takes the vendor, model and key from a connection, so changing vendor is a + connection edit. + - The vendor's, such as ``openai`` or ``anthropic``; the linked page lists each + - 2.11 + - Stable + * - :doc:`Self-hosted models <self_hosted_models>` + - Runs tasks against a model server you host, such as Ollama or vLLM. + - ``openai`` + - 2.11 + - Stable + * - :doc:`Decision models <decision_models>` + - Uses a model that cannot write text, such as TypeSafe's hosted Jev or Ollama's + decision models, to answer a typed question with one value from a set you + define, plus a confidence. + - Jev: ``typesafe``, and install ``pydantic-ai-slim`` 2.45+; others: install + ``pydantic-ai-slim`` 2.53+ + - 2.11 + - Experimental + * - :doc:`LlamaIndex models <hooks/llamaindex>` + - Builds LlamaIndex language and embedding models from a connection. + - ``llamaindex``, and install the vendor's package for vendors other than OpenAI + - 2.11 + - Experimental + +Give an agent tools +------------------- + +:doc:`toolsets/index` compares what each toolset reaches and how you limit it; +:doc:`agent_security` covers what an agent holding them can reach, with a production +checklist. + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`HookToolset <toolsets/hook>` + - Turns the methods you list on any Airflow hook into tools. + - None + - 2.11 + - Stable + * - :doc:`SQLToolset <toolsets/sql>` + - Queries a database, read-only by default, and only the tables in + ``allowed_tables`` when you set it. + - ``sql`` + - 2.11 + - Stable + * - :doc:`ObjectStorageToolset <toolsets/object_storage>` + - Lists and reads files under one object storage path. + - None + - 2.11 + - Stable + * - :doc:`MCPToolset <toolsets/mcp>` + - Gives the agent the tools of an MCP server. + - ``mcp`` + - 2.11 + - Stable + * - :doc:`DataFusionToolset <toolsets/datafusion>` + - Runs SQL over Parquet, CSV and Avro files and Iceberg tables, in the worker. + - ``sql``, plus ``apache-airflow-providers-common-sql[datafusion]``, and its + ``apache.iceberg`` extra for Iceberg + - 2.11 + - Experimental + * - :doc:`AgentSkillsToolset <toolsets/skills>` + - Loads ``SKILL.md`` instruction bundles when the model asks for one. + - ``skills``; on Airflow 2, install ``pydantic-ai-skills`` + - 2.11; the ``skills`` extra 3.0 + - Experimental + * - :doc:`SandboxToolset <sandbox/index>` + - Gives the agent a shell and a filesystem in an isolated sandbox: a hosted + service, a server you run, or a microVM on your machine for local development. + - ``modal``, ``islo``, ``opensandbox`` or ``boat``, by backend; none for ``sbx`` + - 2.11; Modal 3.0 + - Experimental + * - :doc:`ManagedAgentToolset <toolsets/managed_agent>` + - Sends a prompt to an agent on Bedrock AgentCore or Vertex AI Agent Engine. + - The Amazon or Google provider's ``common.ai`` extra + - 2.11 + - Experimental + +Control output and spend +------------------------ + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Structured output <structured_output>` + - Returns a Pydantic object instead of text, on ``@task.llm``, + ``@task.llm_file_analysis`` and ``@task.agent``. Downstream tasks get the object + on 3.3 and later unless ``serialize_output=True``, and a ``dict`` before 3.3. + - None + - 2.11 + - Stable + * - :ref:`Usage limits <agent-usage-budget>` + - Fails the task past a cap on requests, tokens, tool calls or estimated cost (for + models pydantic-ai can price), on ``@task.agent`` and every ``@task.llm*`` task + except ``@task.llm_batch``. On 3.3 and later one total covers every retry. + - None + - 2.11 + - Stable + +Shape an agent run +------------------ + +These configure ``@task.agent`` and ``AgentOperator``. + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Message history <message_history>` + - Continues a conversation from an earlier run. + - None + - 2.11 + - Stable + * - :doc:`Capabilities and guardrails <capabilities>` + - Turns on model reasoning (``Thinking``), web search (``WebSearch``) and other + Pydantic AI capabilities, and input and output guards. + - ``shields`` for guards + - 2.11 + - Stable; ``shields`` guards experimental + * - :doc:`Code mode <code_mode>` + - Lets the model call several tools from one Python snippet. + - ``code-mode`` + - 2.11 + - Experimental + * - :doc:`Agent memory <memory>` + - Pydantic AI's ``Memory`` capability keeps notes the agent writes in one Dag run + and reads in the next. + - None; install ``pydantic-ai-harness`` + - 2.11 + - Third-party (``pydantic-ai-harness``, pre-1.0) + * - :ref:`Prompt caching <agent-prompt-caching>` + - Marks the repeated start of each request for caching on Anthropic, and on the + Bedrock and OpenRouter models that support it, so later requests pay less for it; + OpenAI and Gemini cache on their own. On by default. + - None + - 2.11 + - Stable + +Put a person in the loop +------------------------ + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Approve outputs <approval_gates>` + - Pauses any ``@task.llm*`` task except ``@task.llm_batch`` until a person approves + or rejects the output, optionally editing it. + - None + - 3.1 + - Stable + * - :ref:`Confidence gate <approval-gates:uncertain>` + - Sends the answer to a person, or fails the task, when the model's confidence is + under a bar you set. Only decision models report a confidence; with a text model + every answer counts as uncertain. On ``@task.llm`` and ``@task.llm_branch``. + - As for decision models + - 2.11; sending to a person 3.1 + - Experimental + * - :doc:`Review agent sessions <hitl_review>` + - Lets a reviewer send a ``@task.agent`` answer back with feedback, round after + round. + - None + - 3.1 + - Stable + * - :doc:`Approve tool calls <tool_approval>` + - Pauses ``@task.agent`` before a marked tool runs, until a person approves or + rejects the call. + - None + - 3.3 + - Stable + +Recover from failures +--------------------- + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Durable execution <durable_execution>` + - With ``durable=True`` on ``@task.agent``, or the ``AirflowDurability`` capability + on your own agent, a retry replays the model and tool calls that finished instead + of paying for them again. Before 3.3 it needs + ``[common.ai] durable_cache_path``. + - None + - 2.11 + - Stable + * - :doc:`Retry policies <retry_policies>` + - Has a model read a failure and decide whether the task retries, and when. Works + on any task. + - None + - 3.3 + - Stable + * - :doc:`Provider fallback <provider_fallback>` + - Moves to the next connection in ``fallback_conn_ids`` when a vendor's API + returns an error. + - None + - 2.11 + - Stable + +Bring your own agent +-------------------- + +The Strands, ADK and other-framework routes get every toolset except ``MCPToolset`` and +``AgentSkillsToolset``. + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Pydantic AI agent <frameworks/pydantic_ai>` + - Gives an agent you run in a ``@task`` a model from a connection and the toolsets. + - None + - 2.11 + - Stable + * - :doc:`Strands Agents <frameworks/strands>` + - Gives a Strands agent the toolsets. Run through ``invoke_durably``, it resumes on + retry; before 3.3 that needs a Strands storage you pass in. + - None; install ``strands-agents`` + - 2.11 + - Experimental + * - :doc:`Google ADK <frameworks/adk>` + - Gives an ADK agent the toolsets. + - None; install ``google-adk`` + - 2.11 + - Experimental + * - :doc:`LangChain <hooks/langchain>` + - Builds LangChain chat and embedding models from a connection, and + :doc:`turns a toolset into LangChain tools <toolsets/langchain>`. + - ``langchain``, and install ``langchain-<vendor>`` + - 2.11 + - Experimental + * - :ref:`Any other framework <howto/frameworks:other>` + - Gives an agent in any framework the toolsets, through a framework-neutral tool + interface. + - None + - 2.11 + - Stable + +See what ran +------------ + +.. list-table:: + :header-rows: 1 + :widths: 24 42 14 8 12 + + * - Feature + - What it does + - Extra + - Airflow + - Stability + * - :doc:`Tracing <observability>` + - Exports model and tool calls as OpenTelemetry spans. + - None + - 2.11 + - Experimental + * - :ref:`AI Model tab <howto/observability:model-tab>` + - Shows the model on the task instance page of ``@task.llm`` and ``@task.agent`` + tasks, and for ``@task.agent`` the token counts and an estimated cost. + - None + - 3.4 + - Experimental .. toctree:: :hidden: diff --git a/providers/common/ai/docs/frameworks/index.rst b/providers/common/ai/docs/frameworks/index.rst index 62daee3fe01..0954233668d 100644 --- a/providers/common/ai/docs/frameworks/index.rst +++ b/providers/common/ai/docs/frameworks/index.rst @@ -159,6 +159,8 @@ Start from the agent you already have: is the only one with step-level durable replay across task retries and a built-in human review step, because both are implemented on top of Pydantic AI. +.. _howto/frameworks:other: + Any other framework ------------------- diff --git a/providers/common/ai/docs/index.rst b/providers/common/ai/docs/index.rst index 33383e29213..9abac2e5f8b 100644 --- a/providers/common/ai/docs/index.rst +++ b/providers/common/ai/docs/index.rst @@ -32,6 +32,8 @@ Start here - :doc:`why_common_ai`: what this provider handles that an agent in a plain ``@task`` leaves to you, and when a plain task is enough. +- :doc:`features`: what the provider can do, with the page, extra, minimum Airflow + version and stability of each feature. - :doc:`quickstart`: install, connect a vendor, run a two-task Dag and check its output. - :doc:`use_cases/index`: ten jobs a data team already has, each with the Dag that does it. - :doc:`model_providers`: which vendors work, and the extra, connection and prefix for each. @@ -107,6 +109,7 @@ waits for the result, use that vendor's provider. :hidden: :caption: Guides + Feature map <features> What you can build <use_cases/index> Connections <connections/index> Models and providers <model_providers> @@ -114,7 +117,6 @@ waits for the result, use that vendor's provider. Operators and decorators <operators/index> Toolsets <toolsets/index> Agent frameworks <frameworks/index> - LLM and agent features <features> Document and RAG pipelines <rag_pipelines> Reliability and operations <operations> diff --git a/providers/common/ai/docs/observability.rst b/providers/common/ai/docs/observability.rst index bd964c7dd12..31dd1c7d4f4 100644 --- a/providers/common/ai/docs/observability.rst +++ b/providers/common/ai/docs/observability.rst @@ -103,7 +103,9 @@ How it works ``@task.llm`` additionally pushes the model name above. The other LLM operators still emit GenAI spans correlated to the task span by nesting, but without the identity attributes, the run join key, or the model name. -* **Model tab.** Separately from tracing, the provider adds an "AI Model" tab to the +* .. _howto/observability:model-tab: + + **Model tab.** Separately from tracing, the provider adds an "AI Model" tab to the task-instance page showing the resolved model name and the ``usage`` XCom above as a readable table, including an estimated cost. It needs Airflow >= 3.4, since it relies on the ``applies_to`` operator scoping core added in that version to show up only for
