This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-agents.git
The following commit(s) were added to refs/heads/main by this push:
new bb9b857 [Feature][docs] add docs for introducing embedding models and
vector stores (#221)
bb9b857 is described below
commit bb9b8579f7143d2ac61f6854c26fa479fe1eb570
Author: Alan Z. <[email protected]>
AuthorDate: Tue Sep 30 02:24:29 2025 -0700
[Feature][docs] add docs for introducing embedding models and vector stores
(#221)
---
docs/content/docs/development/embed_and_vector.md | 45 ----
docs/content/docs/development/embedding_models.md | 278 +++++++++++++++++++
.../docs/development/integrate_with_flink.md | 2 +-
docs/content/docs/development/tool_use.md | 2 +-
docs/content/docs/development/vector_stores.md | 297 +++++++++++++++++++++
5 files changed, 577 insertions(+), 47 deletions(-)
diff --git a/docs/content/docs/development/embed_and_vector.md
b/docs/content/docs/development/embed_and_vector.md
deleted file mode 100644
index 1e5aa73..0000000
--- a/docs/content/docs/development/embed_and_vector.md
+++ /dev/null
@@ -1,45 +0,0 @@
----
-title: Embed and Vector Search
-weight: 4
-type: docs
----
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-
-## Embedding Model
-
-{{< hint warning >}}
-**TODO**: What is Embedding Model. How to define and use
EmbedingModelConnection and EmbedingModelSetup.
-
-{{< /hint >}}
-
-{{< hint warning >}}
-**TODO**: List of all built-in Embedding Model and configuration.
-{{< /hint >}}
-
-## Vector Store
-
-{{< hint warning >}}
-**TODO**: What is Vector Store. How to define and use VectorStoreConnection
and VectorStoreSetup.
-{{< /hint >}}
-
-{{< hint warning >}}
-**TODO**: How to use a Vector Store. List of all built-in Vector Store and
configuration.
-{{< /hint >}}
\ No newline at end of file
diff --git a/docs/content/docs/development/embedding_models.md
b/docs/content/docs/development/embedding_models.md
new file mode 100644
index 0000000..046d7a1
--- /dev/null
+++ b/docs/content/docs/development/embedding_models.md
@@ -0,0 +1,278 @@
+---
+title: Embedding Models
+weight: 4
+type: docs
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Embedding Models
+
+{{< hint info >}}
+Embedding models are currently supported in the Python API only. Java API
support is planned for future releases.
+{{< /hint >}}
+
+{{< hint info >}}
+This page covers text-based embedding models. Flink agents does not currently
support multimodal embeddings.
+{{< /hint >}}
+
+## Overview
+
+Embedding models convert text strings into high-dimensional vectors that
capture semantic meaning, enabling powerful semantic search and retrieval
capabilities. These vector representations allow agents to understand and work
with text similarity, semantic search, and knowledge retrieval patterns.
+
+In Flink Agents, embedding models are essential for:
+- **Semantic Search**: Finding relevant documents or information based on
meaning rather than exact keyword matches
+- **Text Similarity**: Measuring how similar two pieces of text are in meaning
+- **Knowledge Retrieval**: Enabling agents to find and retrieve relevant
context from large knowledge bases
+- **Vector Databases**: Storing and querying embeddings for efficient
similarity search
+
+## Getting Started
+
+To use embedding models in your agents, you need to define both a connection
and setup using decorators, then access the embedding model through the runtime
context.
+
+### Resource Decorators
+
+Flink Agents provides decorators to simplify embedding model setup within
agents:
+
+#### @embedding_model_connection
+
+The `@embedding_model_connection` decorator marks a method that creates an
embedding model connection.
+
+#### @embedding_model_setup
+
+The `@embedding_model_setup` decorator marks a method that creates an
embedding model setup.
+
+### Usage Example
+
+Here's how to define and use embedding models in your agent:
+
+```python
+class MyAgent(Agent):
+
+ @embedding_model_connection
+ @staticmethod
+ def openai_connection() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelConnection,
+ api_key="your-api-key-here",
+ base_url="https://api.openai.com/v1";,
+ request_timeout=30.0
+ )
+
+ @embedding_model_setup
+ @staticmethod
+ def openai_embedding() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelSetup,
+ connection="openai_connection",
+ model="your-embedding-model-here"
+ )
+
+ @action(InputEvent)
+ @staticmethod
+ def process_text(event: InputEvent, ctx: RunnerContext) -> None:
+ # Get the embedding model from the runtime context
+ embedding_model = ctx.get_resource("openai_embedding",
ResourceType.EMBEDDING_MODEL)
+
+ # Use the embedding model to generate embeddings
+ user_query = str(event.input)
+ embedding = embedding_model.embed(user_query)
+
+ # Handle the embedding
+ # Process the embedding vector as needed for your use case
+```
+
+## Built-in Providers
+
+### Ollama
+
+Ollama provides local embedding models that run on your machine, offering
privacy and control over your data.
+
+#### Prerequisites
+
+1. Install Ollama from [https://ollama.com/](https://ollama.com/)
+2. Start the Ollama server: `ollama serve`
+3. Download an embedding model: `ollama pull nomic-embed-text`
+
+#### OllamaEmbeddingModelConnection Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `base_url` | str | `"http://localhost:11434"` | Ollama server URL |
+| `request_timeout` | float | `30.0` | HTTP request timeout in seconds |
+
+#### OllamaEmbeddingModelSetup Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `connection` | str | Required | Reference to connection method name |
+| `model` | str | Required | Name of the embedding model to use |
+| `truncate` | bool | `True` | Whether to truncate text exceeding model limits
|
+| `keep_alive` | str/float | `"5m"` | How long to keep model loaded in memory |
+| `additional_kwargs` | dict | `{}` | Additional Ollama API parameters |
+
+#### Usage Example
+
+```python
+class MyAgent(Agent):
+
+ @embedding_model_connection
+ @staticmethod
+ def ollama_connection() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OllamaEmbeddingModelConnection,
+ base_url="http://localhost:11434";,
+ request_timeout=30.0
+ )
+
+ @embedding_model_setup
+ @staticmethod
+ def ollama_embedding() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OllamaEmbeddingModelSetup,
+ connection="ollama_connection",
+ model="nomic-embed-text",
+ truncate=True,
+ keep_alive="5m"
+ )
+
+ ...
+```
+
+#### Available Models
+
+Visit the [Ollama Embedding Models
Library](https://ollama.com/search?c=embedding) for the complete and up-to-date
list of available embedding models.
+
+Some popular options include:
+- **nomic-embed-text**
+- **all-minilm**
+- **mxbai-embed-large**
+
+{{< hint warning >}}
+Model availability and specifications may change. Always check the official
Ollama documentation for the latest information before implementing in
production.
+{{< /hint >}}
+
+### OpenAI
+
+OpenAI provides cloud-based embedding models with state-of-the-art performance.
+
+#### Prerequisites
+
+1. Get an API key from [OpenAI Platform](https://platform.openai.com/)
+
+#### Usage Example
+
+```python
+class MyAgent(Agent):
+
+ @embedding_model_connection
+ @staticmethod
+ def openai_connection() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelConnection,
+ api_key="your-api-key-here",
+ base_url="https://api.openai.com/v1";,
+ request_timeout=30.0,
+ max_retries=3
+ )
+
+ @embedding_model_setup
+ @staticmethod
+ def openai_embedding() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelSetup,
+ connection="openai_connection",
+ model="your-embedding-model-here",
+ encoding_format="float"
+ )
+```
+
+#### OpenAIEmbeddingModelConnection Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `api_key` | str | Required | OpenAI API key for authentication |
+| `base_url` | str | `"https://api.openai.com/v1"` | OpenAI API base URL |
+| `request_timeout` | float | `30.0` | HTTP request timeout in seconds |
+| `max_retries` | int | `3` | Maximum number of retry attempts |
+| `organization` | str | None | Optional organization ID |
+| `project` | str | None | Optional project ID |
+
+#### OpenAIEmbeddingModelSetup Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `connection` | str | Required | Reference to connection method name |
+| `model` | str | Required | OpenAI embedding model name |
+| `encoding_format` | str | `"float"` | Return format ("float" or "base64") |
+| `dimensions` | int | None | Output dimensions (text-embedding-3 models only)
|
+| `user` | str | None | End-user identifier for monitoring |
+| `additional_kwargs` | dict | `{}` | Additional parameters for the OpenAI
embeddings API |
+
+#### Available Models
+
+Visit the [OpenAI Embeddings
documentation](https://platform.openai.com/docs/guides/embeddings#embedding-models)
for the complete and up-to-date list of available embedding models.
+
+Current popular models include:
+- **text-embedding-3-small**
+- **text-embedding-3-large**
+- **text-embedding-ada-002**
+
+{{< hint warning >}}
+Model availability and specifications may change. Always check the official
OpenAI documentation for the latest information before implementing in
production.
+{{< /hint >}}
+
+## Custom Providers
+
+{{< hint warning >}}
+The custom provider APIs are experimental and unstable, subject to
incompatible changes in future releases.
+{{< /hint >}}
+
+If you want to use embedding models not offered by the built-in providers, you
can extend the base embedding classes and implement your own! The embedding
system is built around two main abstract classes:
+
+### BaseEmbeddingModelConnection
+
+Handles the connection to embedding services and provides the core embedding
functionality.
+
+```python
+class MyEmbeddingConnection(BaseEmbeddingModelConnection):
+
+ def embed(self, text: str, **kwargs) -> list[float]:
+ # Core method: convert text to embedding vector
+ # - text: Input text to embed
+ # - kwargs: Additional parameters from model_kwargs
+ # - Returns: List of float values representing the embedding
+ pass
+```
+
+### BaseEmbeddingModelSetup
+
+The setup class acts as a high-level configuration interface that defines
which connection to use and how to configure the embedding model.
+
+
+```python
+class MyEmbeddingSetup(BaseEmbeddingModelSetup):
+ # Add your custom configuration fields here
+
+ @property
+ def model_kwargs(self) -> Dict[str, Any]:
+ # Return model-specific configuration passed to embed()
+ # This dictionary is passed as **kwargs to the embed() method
+ return {"model": self.model, ...}
+```
\ No newline at end of file
diff --git a/docs/content/docs/development/integrate_with_flink.md
b/docs/content/docs/development/integrate_with_flink.md
index 4de85ef..6b1b65f 100644
--- a/docs/content/docs/development/integrate_with_flink.md
+++ b/docs/content/docs/development/integrate_with_flink.md
@@ -1,6 +1,6 @@
---
title: Integrate with Flink
-weight: 6
+weight: 7
type: docs
---
<!--
diff --git a/docs/content/docs/development/tool_use.md
b/docs/content/docs/development/tool_use.md
index 4d7679c..9afa2b5 100644
--- a/docs/content/docs/development/tool_use.md
+++ b/docs/content/docs/development/tool_use.md
@@ -1,6 +1,6 @@
---
title: Tool Use
-weight: 5
+weight: 6
type: docs
---
<!--
diff --git a/docs/content/docs/development/vector_stores.md
b/docs/content/docs/development/vector_stores.md
new file mode 100644
index 0000000..793fdec
--- /dev/null
+++ b/docs/content/docs/development/vector_stores.md
@@ -0,0 +1,297 @@
+---
+title: Vector Stores
+weight: 5
+type: docs
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Vector Stores
+
+{{< hint info >}}
+Vector stores are currently supported in the Python API only. Java API support
is planned for future releases.
+{{< /hint >}}
+
+{{< hint info >}}
+This page covers semantic search using vector stores. Additional query modes
(keyword, hybrid) are planned for future releases.
+{{< /hint >}}
+
+## Overview
+
+Vector stores enable efficient storage, indexing, and retrieval of
high-dimensional embedding vectors alongside their associated documents. They
provide the foundation for semantic search capabilities in AI applications by
allowing fast similarity searches across large document collections.
+
+In Flink Agents, vector stores are essential for:
+- **Document Retrieval**: Finding relevant documents based on semantic
similarity
+- **Knowledge Base Search**: Querying large collections of information using
natural language
+- **Retrieval-Augmented Generation (RAG)**: Providing context to language
models from vector-indexed knowledge
+- **Semantic Similarity**: Comparing and ranking documents by meaning rather
than keywords
+
+## Getting Started
+
+To use vector stores in your agents, you need to configure both a vector store
and an embedding model, then perform semantic search using structured queries.
+
+### Resource Decorators
+
+Flink Agents provides decorators to simplify vector store setup within agents:
+
+#### @vector_store
+
+The `@vector_store` decorator marks a method that creates a vector store.
Vector stores automatically integrate with embedding models for text-based
search.
+
+### Query Objects
+
+Vector stores use structured query objects for consistent interfaces:
+
+```python
+# Create a semantic search query
+query = VectorStoreQuery(
+ mode=VectorStoreQueryMode.SEMANTIC,
+ query_text="What is Apache Flink Agents?",
+ limit=3
+)
+```
+
+### Query Results
+
+When you execute a query, you receive a `VectorStoreQueryResult` object that
contains the search results:
+
+```python
+# Execute the query
+result = vector_store.query(query)
+```
+
+The `VectorStoreQueryResult` contains:
+- **documents**: A list of `Document` objects representing the retrieved
results
+- Each `Document` has:
+ - **content**: The actual text content of the document
+ - **metadata**: Associated metadata (source, category, timestamp, etc.)
+ - **id**: Unique identifier of the document (if available)
+
+### Usage Example
+
+Here's how to define and use vector stores in your agent:
+
+```python
+class MyAgent(Agent):
+
+ # Embedding model setup (required for vector store)
+ @embedding_model_connection
+ @staticmethod
+ def openai_connection() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelConnection,
+ api_key="your-api-key-here"
+ )
+
+ @embedding_model_setup
+ @staticmethod
+ def openai_embedding() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelSetup,
+ connection="openai_connection",
+ model="your-embedding-model-here"
+ )
+
+ # In-memory Chroma setup
+ @vector_store
+ @staticmethod
+ def chroma_store() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=ChromaVectorStore,
+ embedding_model="openai_embedding",
+ collection="my_chroma_store"
+ )
+
+ @action(InputEvent)
+ @staticmethod
+ def search_documents(event: InputEvent, ctx: RunnerContext) -> None:
+ # Get the vector store from the runtime context
+ vector_store = ctx.get_resource("chroma_store",
ResourceType.VECTOR_STORE)
+
+ # Create a semantic search query
+ user_query = str(event.input)
+ query = VectorStoreQuery(
+ mode=VectorStoreQueryMode.SEMANTIC,
+ query_text=user_query,
+ limit=3
+ )
+
+ # Perform the search
+ result = vector_store.query(query)
+
+ # Handle the VectorStoreQueryResult
+ # Process the retrieved context as needed for your use case
+```
+
+## Built-in Providers
+
+### Chroma
+
+[Chroma](https://www.trychroma.com/home) is an open-source vector database
that provides efficient storage and querying of embeddings with support for
multiple deployment modes.
+
+#### Prerequisites
+
+1. Install ChromaDB: `pip install chromadb`
+2. For server mode, start ChromaDB server: `chroma run --path /db_path`
+3. For cloud mode, get API key from [ChromaDB
Cloud](https://www.trychroma.com/)
+
+#### ChromaVectorStore Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `embedding_model` | str | Required | Reference to embedding model method
name |
+| `persist_directory` | str | None | Directory for persistent storage. If
None, uses in-memory client |
+| `host` | str | None | Host for ChromaDB server connection |
+| `port` | int | `8000` | Port for ChromaDB server connection |
+| `api_key` | str | None | API key for Chroma Cloud connection |
+| `client_settings` | Settings | None | ChromaDB client settings for advanced
configuration |
+| `tenant` | str | `"default_tenant"` | ChromaDB tenant for multi-tenancy
support |
+| `database` | str | `"default_database"` | ChromaDB database name |
+| `collection` | str | `"flink_agents_chroma_collection"` | Name of the
ChromaDB collection to use |
+| `collection_metadata` | dict | `{}` | Metadata for the collection |
+| `create_collection_if_not_exists` | bool | `True` | Whether to create the
collection if it doesn't exist |
+
+#### Usage Example
+
+```python
+class MyAgent(Agent):
+
+ # Embedding model setup (required for vector store)
+ @embedding_model_connection
+ @staticmethod
+ def openai_connection() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelConnection,
+ api_key="your-api-key-here"
+ )
+
+ @embedding_model_setup
+ @staticmethod
+ def openai_embedding() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=OpenAIEmbeddingModelSetup,
+ connection="openai_connection",
+ model="your-embedding-model-here"
+ )
+
+ # Vector store setup
+ @vector_store
+ @staticmethod
+ def chroma_store() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=ChromaVectorStore,
+ embedding_model="openai_embedding",
+ persist_directory="/path/to/chroma/data", # For persistent storage
+ collection="my_documents",
+ create_collection_if_not_exists=True
+ # Or use other modes:
+ # "host": "localhost", "port": 8000 # For server mode
+ # "api_key": "your-chroma-cloud-key" # For cloud mode
+ )
+
+ ...
+```
+
+#### Deployment Modes
+
+ChromaDB supports multiple deployment modes:
+
+**In-Memory Mode**
+```python
+@vector_store
+@staticmethod
+def chroma_store() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=ChromaVectorStore,
+ embedding_model="your_embedding_model",
+ collection="my_documents"
+ # No connection configuration needed for in-memory mode
+ )
+```
+
+**Persistent Mode**
+```python
+@vector_store
+@staticmethod
+def chroma_store() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=ChromaVectorStore,
+ embedding_model="your_embedding_model",
+ persist_directory="/path/to/chroma/data",
+ collection="my_documents"
+ )
+```
+
+**Server Mode**
+```python
+@vector_store
+@staticmethod
+def chroma_store() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=ChromaVectorStore,
+ embedding_model="your_embedding_model",
+ host="your-chroma-server.com",
+ port=8000,
+ collection="my_documents"
+ )
+```
+
+**Cloud Mode**
+```python
+@vector_store
+@staticmethod
+def chroma_store() -> ResourceDescriptor:
+ return ResourceDescriptor(
+ clazz=ChromaVectorStore,
+ embedding_model="your_embedding_model",
+ api_key="your-chroma-cloud-api-key",
+ collection="my_documents"
+ )
+```
+
+## Custom Providers
+
+{{< hint warning >}}
+The custom provider APIs are experimental and unstable, subject to
incompatible changes in future releases.
+{{< /hint >}}
+
+If you want to use vector stores not offered by the built-in providers, you
can extend the base vector store class and implement your own! The vector store
system is built around the `BaseVectorStore` abstract class.
+
+### BaseVectorStore
+
+The base class handles text-to-vector conversion and provides the high-level
query interface. You only need to implement the core vector search
functionality.
+
+```python
+class MyVectorStore(BaseVectorStore):
+ # Add your custom configuration fields here
+
+ @property
+ def store_kwargs(self) -> Dict[str, Any]:
+ # Return vector store-specific configuration
+ # These parameters are merged with query-specific parameters
+ return {"index": "my_index", ...}
+
+ def query_embedding(self, embedding: List[float], limit: int = 10,
**kwargs: Any) -> List[Document]:
+ # Core method: perform vector search using pre-computed embedding
+ # - embedding: Pre-computed embedding vector for semantic search
+ # - limit: Maximum number of results to return
+ # - kwargs: Vector store-specific parameters
+ # - Returns: List of Document objects matching the search criteria
+ pass
+```
\ No newline at end of file
