This is an automated email from the ASF dual-hosted git repository.
Lee-W pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new f1f02f9ef9c Document the common-ai LangChain and LlamaIndex connection
types (#71818)
f1f02f9ef9c is described below
commit f1f02f9ef9cb8de5a055a5b6b92d029a97d97673
Author: Wei Lee <[email protected]>
AuthorDate: Wed Aug 19 15:43:45 2026 +0800
Document the common-ai LangChain and LlamaIndex connection types (#71818)
---
providers/common/ai/docs/connections/langchain.rst | 122 ++++++++++++++++++++
.../common/ai/docs/connections/llamaindex.rst | 124 +++++++++++++++++++++
providers/common/ai/docs/index.rst | 2 +
3 files changed, 248 insertions(+)
diff --git a/providers/common/ai/docs/connections/langchain.rst
b/providers/common/ai/docs/connections/langchain.rst
new file mode 100644
index 00000000000..f70e165f758
--- /dev/null
+++ b/providers/common/ai/docs/connections/langchain.rst
@@ -0,0 +1,122 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. _howto/connection:langchain:
+
+LangChain Connection
+====================
+
+The ``langchain`` connection type configures access to LLM providers via
+`LangChain <https://python.langchain.com/>`__'s universal
+``init_chat_model`` / ``init_embeddings`` entry points. It backs
+:class:`~airflow.providers.common.ai.hooks.langchain.LangChainHook` (see
+:doc:`../hooks/langchain` for hook usage and installation instructions).
+
+Default Connection IDs
+----------------------
+
+The ``LangChainHook`` uses ``langchain_default`` by default.
+
+Configuring the Connection
+---------------------------
+
+Chat Model (Extra field)
+ Chat model identifier in ``provider:name`` format, dispatched via
+ ``langchain.chat_models.init_chat_model`` (e.g. ``openai:gpt-4o``,
+ ``anthropic:claude-sonnet-5``). This field appears as a dedicated input
+ in the connection form (via ``conn-fields``) and stores its value in
+ ``extra["model"]``.
+
+Embedding Model (Extra field)
+ Embedding model identifier in ``provider:name`` format, dispatched via
+ ``langchain.embeddings.init_embeddings`` (e.g.
+ ``openai:text-embedding-3-small``). This field appears as a dedicated
+ input in the connection form (via ``conn-fields``) and stores its value
+ in ``extra["embed_model"]``.
+
+ The connection-type definition documents ``cohere:embed-english-v3.0``
+ as an example of the ``provider:name`` format, but the hook only forwards
+ ``api_key`` / ``base_url`` to ``init_embeddings`` -- vendors with bespoke
+ embedding auth such as Cohere are not covered by this connection type yet
+ (see :ref:`Supported providers <langchain-supported-providers>` below and
+ :doc:`../hooks/langchain`).
+
+API Key (Password field)
+ The API key for your LLM provider, passed as ``api_key=`` to
+ ``init_chat_model`` / ``init_embeddings``.
+
+Host (optional)
+ Optional base URL, passed as ``base_url=`` (custom OpenAI-compatible
+ endpoints, Ollama, vLLM).
+
+The ``schema``, ``port``, and ``login`` fields are hidden in the connection
+form; they are not used by this connection type.
+
+.. _langchain-supported-providers:
+
+Supported providers
+--------------------
+
+Only OpenAI-compatible providers work with this hook's ``api_key`` +
+optional ``base_url`` credential surface: OpenAI, Anthropic, Groq,
+Mistral AI, DeepSeek, Ollama, and vLLM. Providers with bespoke auth (AWS
+Bedrock, Google Vertex AI / GenAI, Azure OpenAI, Cohere, HuggingFace) reject
+these kwargs and are not usable through this connection type.
+
+Model resolution order
+-----------------------
+
+Both ``get_chat_model()`` and ``get_embedding_model()`` resolve the model
+identifier from, in order:
+
+1. The ``llm_model`` / ``embed_model`` constructor argument on
``LangChainHook``.
+2. ``extra["model"]`` / ``extra["embed_model"]`` on the connection.
+
+If neither is set, the hook raises a ``ValueError`` when the model is needed.
+
+Examples
+--------
+
+**OpenAI (chat and embeddings)**
+
+.. code-block:: json
+
+ {
+ "conn_type": "langchain",
+ "password": "sk-...",
+ "extra": "{\"model\": \"openai:gpt-4o\", \"embed_model\":
\"openai:text-embedding-3-small\"}"
+ }
+
+**Anthropic (chat only)**
+
+.. code-block:: json
+
+ {
+ "conn_type": "langchain",
+ "password": "sk-ant-...",
+ "extra": "{\"model\": \"anthropic:claude-sonnet-5\"}"
+ }
+
+**Ollama (local, custom endpoint)**
+
+.. code-block:: json
+
+ {
+ "conn_type": "langchain",
+ "host": "http://localhost:11434/v1";,
+ "extra": "{\"model\": \"ollama:llama3\"}"
+ }
diff --git a/providers/common/ai/docs/connections/llamaindex.rst
b/providers/common/ai/docs/connections/llamaindex.rst
new file mode 100644
index 00000000000..82482ce388e
--- /dev/null
+++ b/providers/common/ai/docs/connections/llamaindex.rst
@@ -0,0 +1,124 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. _howto/connection:llamaindex:
+
+LlamaIndex Connection
+======================
+
+The ``llamaindex`` connection type configures access to LLM and embedding
+providers for `LlamaIndex <https://docs.llamaindex.ai/>`__. It backs
+:class:`~airflow.providers.common.ai.hooks.llamaindex.LlamaIndexHook` (see
+:doc:`../hooks/llamaindex` for hook usage and installation instructions).
+
+Default Connection IDs
+----------------------
+
+The ``LlamaIndexHook`` uses ``llamaindex_default`` by default.
+
+Configuring the Connection
+---------------------------
+
+Embedding Model (Extra field)
+ Default LlamaIndex embedding model name (e.g. ``text-embedding-3-small``).
+ This field appears as a dedicated input in the connection form
+ (via ``conn-fields``) and stores its value in ``extra["embed_model"]``.
+
+LLM Model (Extra field)
+ Default LlamaIndex LLM model name (e.g. ``gpt-4o``). This field appears
+ as a dedicated input in the connection form (via ``conn-fields``) and
+ stores its value in ``extra["llm_model"]``.
+
+API Key (Password field)
+ The API key for your LLM/embedding provider, passed as ``api_key=`` to
+ the LlamaIndex model constructor.
+
+Host (optional)
+ Optional base URL, passed as ``api_base=`` (for example, to point at an
+ OpenAI-compatible proxy that serves official OpenAI model names).
+
+The ``schema``, ``port``, and ``login`` fields are hidden in the connection
+form; they are not used by this connection type.
+
+OpenAI models only, BYO for other vendors
+------------------------------------------
+
+``LlamaIndexHook.get_embedding_model()`` always returns an ``OpenAIEmbedding``
+instance, and ``get_llm()`` always returns an ``OpenAI`` LLM instance,
+regardless of the ``host`` you set. Setting ``host`` to point at a different
+server does not relax any validation -- each class validates the model name
+against its own built-in list: a chat/completion-model list
+(``ALL_AVAILABLE_MODELS``, e.g. ``gpt-4o``) for ``OpenAI``, and a separate,
+much smaller embedding-model list
+(``OpenAIEmbeddingModelType``, e.g. ``text-embedding-3-small``) for
+``OpenAIEmbedding``. The two lists mostly do not overlap -- current-generation
+names such as ``gpt-4o`` or ``text-embedding-3-small`` are only valid for one
+of the two classes -- though a handful of legacy names (``ada``, ``babbage``,
+``curie``, ``davinci``) happen to appear in both. The classes differ only in
+*when* their respective check runs:
+
+* ``OpenAIEmbedding`` validates the model name in its constructor, so
+ ``get_embedding_model()`` raises immediately for a name not in its list.
+* ``OpenAI`` (the LLM class) accepts any model name string at construction
+ time, but validates it lazily on first use, inside its ``metadata``
+ property. Any call that touches ``metadata`` -- including ``.chat()`` and
+ ``.complete()`` -- raises a ``ValueError`` for a name not in its list.
+ There is no constructor argument on either class that overrides this
+ check (no ``context_window=`` / ``is_chat_model=`` argument).
+
+In practice this means local or self-hosted models (Ollama, vLLM, and
+similar) are not usable through this connection type, even via ``host=``,
+unless the server is configured to answer to an official OpenAI model name.
+For other vendors and for local models, instantiate the LlamaIndex class
+directly in your ``@task`` and pass it to the operator's ``embed_model=`` /
+``llm=`` parameter -- this bypasses the hook and this connection type
+entirely (see :doc:`../hooks/llamaindex`).
+
+Model resolution order
+-----------------------
+
+Both ``get_embedding_model()`` and ``get_llm()`` resolve the model
+identifier from, in order:
+
+1. The ``embed_model`` / ``llm_model`` constructor argument on
+ ``LlamaIndexHook``.
+2. ``extra["embed_model"]`` / ``extra["llm_model"]`` on the connection.
+
+If neither is set, the hook raises a ``ValueError`` when the model is needed.
+
+Examples
+--------
+
+**OpenAI (embeddings and LLM)**
+
+.. code-block:: json
+
+ {
+ "conn_type": "llamaindex",
+ "password": "sk-...",
+ "extra": "{\"embed_model\": \"text-embedding-3-small\", \"llm_model\":
\"gpt-4o\"}"
+ }
+
+**LLM only (embeddings unset)**
+
+.. code-block:: json
+
+ {
+ "conn_type": "llamaindex",
+ "password": "sk-...",
+ "extra": "{\"llm_model\": \"gpt-4o\"}"
+ }
diff --git a/providers/common/ai/docs/index.rst
b/providers/common/ai/docs/index.rst
index 1166c970296..c34c3a70932 100644
--- a/providers/common/ai/docs/index.rst
+++ b/providers/common/ai/docs/index.rst
@@ -149,6 +149,8 @@ See the Optional dependencies table below for the exact
package each extra insta
Quick start <quickstart>
Connection types <connections/pydantic_ai>
MCP connection <connections/mcp>
+ LangChain connection <connections/langchain>
+ LlamaIndex connection <connections/llamaindex>
Hooks <hooks/index>
Toolsets <toolsets>
Operators <operators/index>
