This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new b3a8ff373c1 [ADD] add ai-overview.md (#3183)
b3a8ff373c1 is described below
commit b3a8ff373c1bf6d57be589a265c69c507b8bc4de
Author: zhuwei <[email protected]>
AuthorDate: Sat Dec 13 12:15:24 2025 +0800
[ADD] add ai-overview.md (#3183)
## Versions
- [ ] dev
- [ ] 4.x
- [ ] 3.x
- [ ] 2.1
## Languages
- [ ] Chinese
- [ ] English
## Docs Checklist
- [ ] Checked by AI
- [ ] Test Cases Built
Co-authored-by: zhuwei <[email protected]>
Co-authored-by: yiguolei <[email protected]>
---
docs/ai/ai-overview.md | 123 +++++++++++++++++++++
.../current/ai/ai-overview.md | 121 ++++++++++++++++++++
.../version-4.x/ai/ai-overview.md | 121 ++++++++++++++++++++
sidebars.ts | 1 +
static/images/vector-search/image-5.png | Bin 0 -> 126266 bytes
versioned_docs/version-4.x/ai/ai-overview.md | 123 +++++++++++++++++++++
versioned_sidebars/version-4.x-sidebars.json | 1 +
7 files changed, 490 insertions(+)
diff --git a/docs/ai/ai-overview.md b/docs/ai/ai-overview.md
new file mode 100644
index 00000000000..3a057909440
--- /dev/null
+++ b/docs/ai/ai-overview.md
@@ -0,0 +1,123 @@
+---
+{
+ "title": "AI Overview",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+
+As AI technologies continue to advance at an unprecedented pace, data
infrastructure has become the cornerstone of modern AI applications. Apache
Doris, a high-performance real-time analytical database, provides native
integration of full-text search, vector search, AI functions, and MCP-based
intelligent interaction. Together, these capabilities form a comprehensive AI
data stack that spans storage, retrieval, and analysis.
+
+- [full-text search](text-search/overview.md)
+- [vector search](vector-search/overview.md)
+- [AI functions](ai-function-overview.md)
+- [Doris MCP Server](https://github.com/apache/doris-mcp-server)
+
+Doris delivers a unified, high-performance, and cost-efficient solution for a
wide range of AI-driven workloads, including hybrid search and analytics, agent
facing data analysis, semantic search, RAG application development, and
observability for large-scale AI systems.
+
+## Agent Facing Analytics
+
+With the rise of AI Agent technology, an increasing number of analytical
decisions will be completed automatically by AI, requiring data platforms to
deliver ultimate real-time performance and high concurrency capabilities.
Unlike traditional "manual analysis," Agent Facing Analytics demands data
queries and decision-making to be completed at millisecond scale, supporting
concurrent access from massive numbers of Agents. Typical scenarios include
real-time fraud detection, intelligent ad [...]
+
+Doris demonstrates outstanding advantages in these agent-facing analytical
scenarios with its high-performance MPP architecture:
+
+- **Real-Time Ingestion & Update**: ensuring Agent decisions are based on the
latest data, ~ 1s minimum data latency
+
+- **Blazing-Fast Analytics**: Average query latency < 100ms, meeting real-time
decision requirements for Agents
+- **High-Concurrent Queries**: Supports 10,000+ QPS, easily handling massive
Agent concurrent queries
+- **Native Agent integration**: Seamlessly integrates with AI Agents through
MCP Server, simplifying development and integration workflows
+
+## Hybrid Search and Analytics Processing
+
+
+
+Semi-structured and unstructured data are becoming first-class citizens in
data analytics. Customer reviews, chat logs, production logs, vehicle signals,
and other data have been deeply integrated into business decision-making
processes. Traditional structured analytics solutions need to incorporate
full-text retrieval and vector search capabilities, supporting semantic search
while enabling multidimensional analysis and aggregation statistics on the same
platform. Examples include:
+
+- **Customer insights**: Combining review text retrieval with user behavior
analysis to precisely identify customer needs and satisfaction trends
+- **Smart manufacturing**: Integrating production log full-text search,
equipment image recognition, and IoT metric analysis to achieve fault
prediction and quality optimization
+- **Internet of Vehicles**: Synthesizing vehicle signal data analysis, user
feedback text mining, and driving behavior vector retrieval to enhance smart
cockpit experiences
+
+Building AI applications for the above scenarios based on Doris's
high-performance real-time analytics, text indexing, and vector indexing
capabilities offers multiple advantages:
+
+- **Unified architecture**: Processes structured analytics, full-text
retrieval, and vector search on a single platform, eliminating data migration
and heterogeneous system integration
+- **Hybrid query performance**: Single SQL executes vector similarity search,
keyword filtering, and aggregation analysis simultaneously with excellent query
performance
+- **Flexible schema support**: VARIANT type natively supports dynamic JSON
structures, Light Schema Change enables second-level field and index
modifications
+- **Full-stack optimization**: End-to-end optimization from inverted indexes
and vector indexes to MPP execution engine, balancing retrieval accuracy and
analytical efficiency
+
+## Lakehouse for AI
+
+AI model and application development requires preparing training sets,
performing feature engineering, and evaluating data quality from massive
datasets. Traditional architectures often require frequent data migration
between data lakes and analytical engines. The Lakehouse architecture deeply
integrates the open storage of data lakes with real-time analytical engines,
supporting the entire workflow of data preparation, feature engineering, and
model evaluation on a unified platform, eli [...]
+
+- **Lakehouse unified architecture**: Builds an open lakehouse based on open
table formats (such as Iceberg/Paimon) and Catalogs, uniformly managing
analytical data and AI data
+- **Real-Time Analytics Engine**: Doris serves as a real-time analytical
engine, supporting interactive queries and lightweight ETL, providing the
fastest SQL computing capabilities for data preparation and feature engineering
+- **Seamless data flow**: Directly reads and writes to data lakes without data
movement, unified management at the storage layer and flexible acceleration at
the compute layer
+
+Lakehouse architecture based on Doris accelerates the entire AI workflow:
+
+- **Large-scale data preparation**: Leveraging Doris's efficient data
processing capabilities to filter, sample, and cleanse data from PB-scale data
lakes, rapidly building high-quality training datasets
+- **Real-time feature engineering**: Utilizing Doris's real-time analytics
capabilities to perform online feature extraction, transformation, and
aggregation computing, providing real-time feature services for model training
and inference
+- **Quality evaluation**: Conducting multidimensional rapid analysis on test
sets and production data, continuously monitoring model performance and data
drift
+
+## RAG (Retrieval-Augmented Generation)
+
+RAG retrieves relevant information from external knowledge bases to provide
context for large models, effectively addressing model hallucination and
knowledge currency issues. The vector engine is a core component of RAG
systems, requiring rapid recall of the most relevant document fragments from
massive knowledge bases while supporting high-concurrency user query requests
to ensure application responsiveness.
+
+- **Enterprise knowledge**: Building intelligent Q&A systems based on internal
documents and manuals, enabling employees to quickly obtain accurate answers
through natural language
+- **Intelligent customer service assistant**: Combining product knowledge
bases and historical cases to provide precise response suggestions for customer
service personnel or chatbots
+- **Intelligent document assistant**: Rapidly locating relevant content in
large-scale document collections to assist research, writing, and
decision-making processes
+
+Building RAG applications based on Doris offers the following advantages in
these scenarios:
+
+- **High concurrency performance**: Distributed architecture supports
high-concurrency vector retrieval, easily handling large-scale concurrent user
access
+- **Hybrid retrieval capability**: Single SQL executes vector similarity
search and keyword filtering simultaneously, balancing semantic recall and
exact matching
+- **Elastic scaling**: Query performance scales linearly with cluster
expansion, seamlessly transitioning from millions to tens of billions of vectors
+- **Unified solution**: Uniformly manages vector data, original documents, and
business data, simplifying the data architecture for RAG applications
+
+## AI Observability
+
+AI model training iterations and application operations generate massive
amounts of logs, metrics, and tracing data. To precisely locate issues and
continuously optimize performance, observability systems have become a critical
component of AI infrastructure. As business scale expands, observability
platforms face multiple challenges including high-throughput writes of PB-scale
data, millisecond-level retrieval response, and cost control. Typical use cases
include:
+
+- **Model training monitoring**: Real-time tracking of training metrics and
resource consumption, rapidly identifying training anomalies and performance
bottlenecks
+- **Inference service tracing**: Recording the complete trace of each
inference request, analyzing latency sources and error patterns
+- **AI** **application log analysis**: Full-text retrieval and aggregation
analysis of massive application logs, supporting troubleshooting and behavioral
insights
+
+Building AI Observability with Doris offers the following advantages:
+
+- **Ultimate performance**: Supports sustained writes of PB/day (10GB/s),
inverted indexes accelerate log retrieval with second-level response
+- **Cost optimization**: Compression ratios of 5:1 to 10:1, storage cost
savings of 50%-80%, supports low-cost storage for cold data
+- **Flexible schema**: Light Schema Change enables second-level field
modifications, VARIANT type natively supports dynamic JSON structures
+- **Ecosystem-friendly**: Compatible with OpenTelemetry and ELK ecosystems,
supports integration with Grafana/Kibana visualization tools
+
+## Semantic Search
+
+Semantic search captures the deep meaning of text through vectorization
techniques. Even when query terms differ from document wording, semantically
relevant content can still be retrieved. This is crucial for scenarios such as
cross-language retrieval, synonym recognition, and intent understanding,
significantly improving search recall rates and user experience. Typical use
cases include:
+
+- **Enterprise document retrieval**: Employees describe issues in natural
language, and the system understands intent to recall semantically relevant
policies, procedures, and knowledge from massive documents
+- **E-commerce product search**: Users input "breathable shoes suitable for
summer," and the system understands the need to recall relevant products rather
than merely matching keywords
+- **Content recommendation**: Intelligent recommendations based on semantic
similarity of articles and videos, discovering content of potential interest
with different wording
+
+Building semantic search applications based on Doris offers the following
advantages:
+
+- **High-performance vector retrieval**: Supports HNSW and IVF algorithms,
sub-second response for hundred-million-scale vectors, easily handling
large-scale semantic search requirements
+- **Enhanced hybrid retrieval**: Single SQL integrates semantic search and
keyword filtering, ensuring necessary vocabulary hits while recalling
semantically relevant content
+- **Multimodal extension**: Supports not only text semantic search but can
also extend to semantic retrieval of multimodal content such as images and audio
+- **Flexible quantization optimization**: Through SQ/PQ quantization
techniques, significantly reduces storage and computing costs while maintaining
retrieval accuracy
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai/ai-overview.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai/ai-overview.md
new file mode 100644
index 00000000000..6fc79e9eba7
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai/ai-overview.md
@@ -0,0 +1,121 @@
+---
+{
+ "title": "AI 概述",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+在 AI 技术快速演进的时代，数据基础设施正成为AI应用的核心支撑。Apache Doris
作为一款高性能、实时分析型数据库，深度融合了文本搜索、向量搜索、AI 函数和MCP智能交互能力，构建从数据存储、检索到分析的完整 AI 数据栈。
+
+- [文本搜索概述](text-search/overview.md)
+- [向量搜索概述](vector-search/overview.md)
+- [AI函数概述](ai-function-overview.md)
+- [Doris MCP Server](https://github.com/apache/doris-mcp-server)
+
+Doris 提供高性能、低成本、易集成的一体化解决方案, 广泛支持混合检索与分析、面向Agent的数据分析、RAG
应用构建、语义检索应用，以及大规模AI系统和应用的可观测性分析等场景。
+
+## Agent Facing Analytics
+
+随着 AI Agent 技术的兴起,越来越多的分析决策将由 AI 自动完成,这要求数据平台具备极致的实时性和高并发能力。与传统"人工分析"不同,Agent
Facing Analytics 需要在毫秒级完成数据查询和决策,支持海量 Agent 的并发访问。典型场景包括实时反欺诈检测、智能广告投放、个性化推荐等。
+
+Doris凭借高性能MPP架构，在这类面向Agent的分析场景中有非常突出的优势：
+
+- 亚秒级数据延迟：支持实时数据摄入与更新,确保 Agent 决策基于最新数据
+- 毫秒级查询响应：平均查询延迟 < 100ms,满足 Agent 实时决策需求
+- 万级 QPS 并发：支持 10,000+ QPS,轻松应对海量 Agent 并发查询
+- 原生 Agent 集成：通过 MCP Server 无缝对接 AI Agent,简化开发集成流程
+
+## Hybrid Search and Analytics Processing
+
+
+
+半结构化、非结构化数据正成为数据分析的一等公民。客户评论、聊天记录、生产日志、车机信号等数据已深度融入业务决策流程。传统的结构化分析方案需要融合全文检索和向量检索能力，在同一平台上既支持语义搜索，又能进行多维分析和聚合统计。例如：
+
+- 客户洞察：结合评论文本检索和用户行为分析，精准定位客户需求和满意度趋势。
+- 智能制造：融合生产日志全文搜索、设备图像识别和 IoT 指标分析，实现故障预测和质量优化。
+- 车联网：综合车机信号数据分析、用户反馈文本挖掘和驾驶行为向量检索，提升智能座舱体验。
+
+基于Doris的高性能实时分析、文本索引和向量索引能力构建上述场景的AI应用，具备多方面的优势：
+
+- 一体化架构：在单一平台统一处理结构化分析、全文检索和向量搜索，无需数据迁移和异构系统集成
+- 混合查询性能：单条 SQL 同时执行向量相似度搜索、关键词过滤和聚合分析，查询性能优异
+- 灵活 Schema 支持：VARIANT 类型原生支持动态 JSON 结构，Light Schema Change 秒级变更字段和索引
+- 全栈优化：从倒排索引、向量索引到 MPP 执行引擎的端到端优化，兼顾检索精度和分析效率
+
+## Lakehouse for AI
+
+AI 模型和应用开发需要从海量数据中准备训练集、进行特征工程、评估数据质量，传统架构往往需要在数据湖和分析引擎间频繁迁移数据。Lakehouse
架构将数据湖的开放存储与实时分析引擎深度融合，在统一平台上支撑数据准备、特征工程和模型评估的全流程，消除数据孤岛，加速 AI 开发迭代。
+
+- 湖仓一体架构：基于开放湖表格式（如Iceberg/Paimon等）和 Catalog 构建开放湖仓，统一管理分析数据和 AI 数据
+- 极速 SQL 引擎：Doris作为实时分析引擎，支持交互式查询和轻量级 ETL，为数据准备和特征工程提供最快的 SQL 计算能力
+- 无缝数据流转：直接读写数据湖，无需数据搬迁，在存储层统一管理，在计算层灵活加速
+
+基于Doris的Lakehouse架构对AI全流程进行加速：
+
+- 大规模数据准备：利用Doris的高效数据处理能力，从 PB 级数据湖中高效过滤、采样和清洗数据，快速构建高质量训练数据集
+- 实时特征工程：利用Doris的实时分析能力，在线进行特征提取、转换和聚合计算，为模型训练和推理提供实时特征服务
+- 质量评估：对测试集和线上数据进行多维度快速分析，持续监控模型表现和数据漂移
+
+## RAG（Retrieval-Augmented Generation）
+
+RAG 通过从外部知识库检索相关信息为大模型提供上下文，有效解决模型幻觉和知识时效性问题。向量引擎是 RAG
系统的核心组件，需要在海量知识库中快速召回最相关的文档片段，同时支持高并发的用户查询请求，确保应用的响应体验。
+
+- 企业知识库：基于内部文档、手册构建智能问答系统，员工通过自然语言快速获取准确答案
+- 智能客服助手：结合产品知识库和历史案例，为客服人员或聊天机器人提供精准的回复建议
+- 智能文档助手：在大规模文档集合中快速定位相关内容，辅助研究、写作和决策过程
+
+在这类场景中，基于Doris构建RAG应用具备以下优势：
+
+- 高并发性能：分布式架构支持高并发向量检索，轻松应对大规模用户并发访问
+- 混合检索能力：在单条 SQL 中同时执行向量相似度搜索和关键词过滤，兼顾语义召回和精确匹配
+- 弹性扩展：随集群扩容线性提升检索性能，从百万到百亿级向量无缝平滑过渡
+- 一体化方案：统一管理向量数据、原始文档和业务数据，简化 RAG 应用的数据架构
+
+## AI Observability
+
+AI 模型训练迭代和应用运行过程中会产生海量日志、指标和追踪数据。为精准定位问题、持续优化性能，可观测性系统成为 AI
基础设施的关键一环。随着业务规模扩张，可观测平台面临 PB 级数据的高吞吐写入、毫秒级检索响应和成本控制的多重挑战。典型用例包括：
+
+- 模型训练监控，实时追踪训练指标、资源消耗，快速定位训练异常和性能瓶颈；
+- 推理服务追踪，记录每次推理请求的完整链路，分析延迟来源和错误模式；
+- AI 应用日志分析，海量应用日志的全文检索和聚合分析，支持故障排查和行为洞察。
+
+使用Doris构建AI Observability，具备以下优势：
+
+- 极致性能：支持 PB 级/天（10GB/s）持续写入，倒排索引加速日志检索，秒级响应
+- 成本优化：压缩率达 5:1 至 10:1，存储成本节省 50%-80%，支持冷数据低成本存储
+- 灵活 Schema：Light Schema Change 秒级变更字段，VARIANT 类型原生支持动态 JSON 结构
+- 生态友好：兼容 OpenTelemetry、ELK 生态，支持对接 Grafana/Kibana 可视化工具
+
+## Semantic Search
+
+语义搜索通过向量化技术捕捉文本深层含义，即使查询词与文档用词不同，也能召回语义相关的内容。这对于跨语言检索、同义词识别、意图理解等场景至关重要，显著提升搜索的召回率和用户体验。典型用例包括：
+
+- 企业文档检索：员工用自然语言描述问题，系统理解意图后从海量文档中召回语义相关的政策、流程和知识
+- 电商商品搜索：用户输入"适合夏天穿的透气鞋子"，系统理解需求并召回相关产品，而非仅匹配关键词
+- 内容推荐平台：基于文章、视频的语义相似度进行智能推荐，发现用户可能感兴趣但用词不同的内
+
+基于Doris构建语义搜索场景的应用，具备以下优势：
+
+- 高性能向量检索：支持 HNSW 和 IVF 算法，亿级向量亚秒级响应，轻松应对大规模语义搜索需求
+- 混合检索增强：单条 SQL 融合语义搜索和关键词过滤，在召回语义相关内容的同时确保必要词汇命中
+- 多模态扩展：不仅支持文本语义搜索，还可扩展至图片、音频等多模态内容的语义检索
+- 灵活量化优化：通过 SQ/PQ 量化技术，在保证检索精度的前提下大幅降低存储和计算成本
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/ai/ai-overview.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/ai/ai-overview.md
new file mode 100644
index 00000000000..6fc79e9eba7
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/ai/ai-overview.md
@@ -0,0 +1,121 @@
+---
+{
+ "title": "AI 概述",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+在 AI 技术快速演进的时代，数据基础设施正成为AI应用的核心支撑。Apache Doris
作为一款高性能、实时分析型数据库，深度融合了文本搜索、向量搜索、AI 函数和MCP智能交互能力，构建从数据存储、检索到分析的完整 AI 数据栈。
+
+- [文本搜索概述](text-search/overview.md)
+- [向量搜索概述](vector-search/overview.md)
+- [AI函数概述](ai-function-overview.md)
+- [Doris MCP Server](https://github.com/apache/doris-mcp-server)
+
+Doris 提供高性能、低成本、易集成的一体化解决方案, 广泛支持混合检索与分析、面向Agent的数据分析、RAG
应用构建、语义检索应用，以及大规模AI系统和应用的可观测性分析等场景。
+
+## Agent Facing Analytics
+
+随着 AI Agent 技术的兴起,越来越多的分析决策将由 AI 自动完成,这要求数据平台具备极致的实时性和高并发能力。与传统"人工分析"不同,Agent
Facing Analytics 需要在毫秒级完成数据查询和决策,支持海量 Agent 的并发访问。典型场景包括实时反欺诈检测、智能广告投放、个性化推荐等。
+
+Doris凭借高性能MPP架构，在这类面向Agent的分析场景中有非常突出的优势：
+
+- 亚秒级数据延迟：支持实时数据摄入与更新,确保 Agent 决策基于最新数据
+- 毫秒级查询响应：平均查询延迟 < 100ms,满足 Agent 实时决策需求
+- 万级 QPS 并发：支持 10,000+ QPS,轻松应对海量 Agent 并发查询
+- 原生 Agent 集成：通过 MCP Server 无缝对接 AI Agent,简化开发集成流程
+
+## Hybrid Search and Analytics Processing
+
+
+
+半结构化、非结构化数据正成为数据分析的一等公民。客户评论、聊天记录、生产日志、车机信号等数据已深度融入业务决策流程。传统的结构化分析方案需要融合全文检索和向量检索能力，在同一平台上既支持语义搜索，又能进行多维分析和聚合统计。例如：
+
+- 客户洞察：结合评论文本检索和用户行为分析，精准定位客户需求和满意度趋势。
+- 智能制造：融合生产日志全文搜索、设备图像识别和 IoT 指标分析，实现故障预测和质量优化。
+- 车联网：综合车机信号数据分析、用户反馈文本挖掘和驾驶行为向量检索，提升智能座舱体验。
+
+基于Doris的高性能实时分析、文本索引和向量索引能力构建上述场景的AI应用，具备多方面的优势：
+
+- 一体化架构：在单一平台统一处理结构化分析、全文检索和向量搜索，无需数据迁移和异构系统集成
+- 混合查询性能：单条 SQL 同时执行向量相似度搜索、关键词过滤和聚合分析，查询性能优异
+- 灵活 Schema 支持：VARIANT 类型原生支持动态 JSON 结构，Light Schema Change 秒级变更字段和索引
+- 全栈优化：从倒排索引、向量索引到 MPP 执行引擎的端到端优化，兼顾检索精度和分析效率
+
+## Lakehouse for AI
+
+AI 模型和应用开发需要从海量数据中准备训练集、进行特征工程、评估数据质量，传统架构往往需要在数据湖和分析引擎间频繁迁移数据。Lakehouse
架构将数据湖的开放存储与实时分析引擎深度融合，在统一平台上支撑数据准备、特征工程和模型评估的全流程，消除数据孤岛，加速 AI 开发迭代。
+
+- 湖仓一体架构：基于开放湖表格式（如Iceberg/Paimon等）和 Catalog 构建开放湖仓，统一管理分析数据和 AI 数据
+- 极速 SQL 引擎：Doris作为实时分析引擎，支持交互式查询和轻量级 ETL，为数据准备和特征工程提供最快的 SQL 计算能力
+- 无缝数据流转：直接读写数据湖，无需数据搬迁，在存储层统一管理，在计算层灵活加速
+
+基于Doris的Lakehouse架构对AI全流程进行加速：
+
+- 大规模数据准备：利用Doris的高效数据处理能力，从 PB 级数据湖中高效过滤、采样和清洗数据，快速构建高质量训练数据集
+- 实时特征工程：利用Doris的实时分析能力，在线进行特征提取、转换和聚合计算，为模型训练和推理提供实时特征服务
+- 质量评估：对测试集和线上数据进行多维度快速分析，持续监控模型表现和数据漂移
+
+## RAG（Retrieval-Augmented Generation）
+
+RAG 通过从外部知识库检索相关信息为大模型提供上下文，有效解决模型幻觉和知识时效性问题。向量引擎是 RAG
系统的核心组件，需要在海量知识库中快速召回最相关的文档片段，同时支持高并发的用户查询请求，确保应用的响应体验。
+
+- 企业知识库：基于内部文档、手册构建智能问答系统，员工通过自然语言快速获取准确答案
+- 智能客服助手：结合产品知识库和历史案例，为客服人员或聊天机器人提供精准的回复建议
+- 智能文档助手：在大规模文档集合中快速定位相关内容，辅助研究、写作和决策过程
+
+在这类场景中，基于Doris构建RAG应用具备以下优势：
+
+- 高并发性能：分布式架构支持高并发向量检索，轻松应对大规模用户并发访问
+- 混合检索能力：在单条 SQL 中同时执行向量相似度搜索和关键词过滤，兼顾语义召回和精确匹配
+- 弹性扩展：随集群扩容线性提升检索性能，从百万到百亿级向量无缝平滑过渡
+- 一体化方案：统一管理向量数据、原始文档和业务数据，简化 RAG 应用的数据架构
+
+## AI Observability
+
+AI 模型训练迭代和应用运行过程中会产生海量日志、指标和追踪数据。为精准定位问题、持续优化性能，可观测性系统成为 AI
基础设施的关键一环。随着业务规模扩张，可观测平台面临 PB 级数据的高吞吐写入、毫秒级检索响应和成本控制的多重挑战。典型用例包括：
+
+- 模型训练监控，实时追踪训练指标、资源消耗，快速定位训练异常和性能瓶颈；
+- 推理服务追踪，记录每次推理请求的完整链路，分析延迟来源和错误模式；
+- AI 应用日志分析，海量应用日志的全文检索和聚合分析，支持故障排查和行为洞察。
+
+使用Doris构建AI Observability，具备以下优势：
+
+- 极致性能：支持 PB 级/天（10GB/s）持续写入，倒排索引加速日志检索，秒级响应
+- 成本优化：压缩率达 5:1 至 10:1，存储成本节省 50%-80%，支持冷数据低成本存储
+- 灵活 Schema：Light Schema Change 秒级变更字段，VARIANT 类型原生支持动态 JSON 结构
+- 生态友好：兼容 OpenTelemetry、ELK 生态，支持对接 Grafana/Kibana 可视化工具
+
+## Semantic Search
+
+语义搜索通过向量化技术捕捉文本深层含义，即使查询词与文档用词不同，也能召回语义相关的内容。这对于跨语言检索、同义词识别、意图理解等场景至关重要，显著提升搜索的召回率和用户体验。典型用例包括：
+
+- 企业文档检索：员工用自然语言描述问题，系统理解意图后从海量文档中召回语义相关的政策、流程和知识
+- 电商商品搜索：用户输入"适合夏天穿的透气鞋子"，系统理解需求并召回相关产品，而非仅匹配关键词
+- 内容推荐平台：基于文章、视频的语义相似度进行智能推荐，发现用户可能感兴趣但用词不同的内
+
+基于Doris构建语义搜索场景的应用，具备以下优势：
+
+- 高性能向量检索：支持 HNSW 和 IVF 算法，亿级向量亚秒级响应，轻松应对大规模语义搜索需求
+- 混合检索增强：单条 SQL 融合语义搜索和关键词过滤，在召回语义相关内容的同时确保必要词汇命中
+- 多模态扩展：不仅支持文本语义搜索，还可扩展至图片、音频等多模态内容的语义检索
+- 灵活量化优化：通过 SQ/PQ 量化技术，在保证检索精度的前提下大幅降低存储和计算成本
\ No newline at end of file
diff --git a/sidebars.ts b/sidebars.ts
index c691fa7ab3a..8f6962c296d 100644
--- a/sidebars.ts
+++ b/sidebars.ts
@@ -393,6 +393,7 @@ const sidebars: SidebarsConfig = {
type: 'category',
label: 'AI',
items: [
+ 'ai/ai-overview',
'ai/ai-function-overview',
{
type: 'category',
diff --git a/static/images/vector-search/image-5.png
b/static/images/vector-search/image-5.png
new file mode 100644
index 00000000000..6a5226101f6
Binary files /dev/null and b/static/images/vector-search/image-5.png differ
diff --git a/versioned_docs/version-4.x/ai/ai-overview.md
b/versioned_docs/version-4.x/ai/ai-overview.md
new file mode 100644
index 00000000000..3a057909440
--- /dev/null
+++ b/versioned_docs/version-4.x/ai/ai-overview.md
@@ -0,0 +1,123 @@
+---
+{
+ "title": "AI Overview",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+
+As AI technologies continue to advance at an unprecedented pace, data
infrastructure has become the cornerstone of modern AI applications. Apache
Doris, a high-performance real-time analytical database, provides native
integration of full-text search, vector search, AI functions, and MCP-based
intelligent interaction. Together, these capabilities form a comprehensive AI
data stack that spans storage, retrieval, and analysis.
+
+- [full-text search](text-search/overview.md)
+- [vector search](vector-search/overview.md)
+- [AI functions](ai-function-overview.md)
+- [Doris MCP Server](https://github.com/apache/doris-mcp-server)
+
+Doris delivers a unified, high-performance, and cost-efficient solution for a
wide range of AI-driven workloads, including hybrid search and analytics, agent
facing data analysis, semantic search, RAG application development, and
observability for large-scale AI systems.
+
+## Agent Facing Analytics
+
+With the rise of AI Agent technology, an increasing number of analytical
decisions will be completed automatically by AI, requiring data platforms to
deliver ultimate real-time performance and high concurrency capabilities.
Unlike traditional "manual analysis," Agent Facing Analytics demands data
queries and decision-making to be completed at millisecond scale, supporting
concurrent access from massive numbers of Agents. Typical scenarios include
real-time fraud detection, intelligent ad [...]
+
+Doris demonstrates outstanding advantages in these agent-facing analytical
scenarios with its high-performance MPP architecture:
+
+- **Real-Time Ingestion & Update**: ensuring Agent decisions are based on the
latest data, ~ 1s minimum data latency
+
+- **Blazing-Fast Analytics**: Average query latency < 100ms, meeting real-time
decision requirements for Agents
+- **High-Concurrent Queries**: Supports 10,000+ QPS, easily handling massive
Agent concurrent queries
+- **Native Agent integration**: Seamlessly integrates with AI Agents through
MCP Server, simplifying development and integration workflows
+
+## Hybrid Search and Analytics Processing
+
+
+
+Semi-structured and unstructured data are becoming first-class citizens in
data analytics. Customer reviews, chat logs, production logs, vehicle signals,
and other data have been deeply integrated into business decision-making
processes. Traditional structured analytics solutions need to incorporate
full-text retrieval and vector search capabilities, supporting semantic search
while enabling multidimensional analysis and aggregation statistics on the same
platform. Examples include:
+
+- **Customer insights**: Combining review text retrieval with user behavior
analysis to precisely identify customer needs and satisfaction trends
+- **Smart manufacturing**: Integrating production log full-text search,
equipment image recognition, and IoT metric analysis to achieve fault
prediction and quality optimization
+- **Internet of Vehicles**: Synthesizing vehicle signal data analysis, user
feedback text mining, and driving behavior vector retrieval to enhance smart
cockpit experiences
+
+Building AI applications for the above scenarios based on Doris's
high-performance real-time analytics, text indexing, and vector indexing
capabilities offers multiple advantages:
+
+- **Unified architecture**: Processes structured analytics, full-text
retrieval, and vector search on a single platform, eliminating data migration
and heterogeneous system integration
+- **Hybrid query performance**: Single SQL executes vector similarity search,
keyword filtering, and aggregation analysis simultaneously with excellent query
performance
+- **Flexible schema support**: VARIANT type natively supports dynamic JSON
structures, Light Schema Change enables second-level field and index
modifications
+- **Full-stack optimization**: End-to-end optimization from inverted indexes
and vector indexes to MPP execution engine, balancing retrieval accuracy and
analytical efficiency
+
+## Lakehouse for AI
+
+AI model and application development requires preparing training sets,
performing feature engineering, and evaluating data quality from massive
datasets. Traditional architectures often require frequent data migration
between data lakes and analytical engines. The Lakehouse architecture deeply
integrates the open storage of data lakes with real-time analytical engines,
supporting the entire workflow of data preparation, feature engineering, and
model evaluation on a unified platform, eli [...]
+
+- **Lakehouse unified architecture**: Builds an open lakehouse based on open
table formats (such as Iceberg/Paimon) and Catalogs, uniformly managing
analytical data and AI data
+- **Real-Time Analytics Engine**: Doris serves as a real-time analytical
engine, supporting interactive queries and lightweight ETL, providing the
fastest SQL computing capabilities for data preparation and feature engineering
+- **Seamless data flow**: Directly reads and writes to data lakes without data
movement, unified management at the storage layer and flexible acceleration at
the compute layer
+
+Lakehouse architecture based on Doris accelerates the entire AI workflow:
+
+- **Large-scale data preparation**: Leveraging Doris's efficient data
processing capabilities to filter, sample, and cleanse data from PB-scale data
lakes, rapidly building high-quality training datasets
+- **Real-time feature engineering**: Utilizing Doris's real-time analytics
capabilities to perform online feature extraction, transformation, and
aggregation computing, providing real-time feature services for model training
and inference
+- **Quality evaluation**: Conducting multidimensional rapid analysis on test
sets and production data, continuously monitoring model performance and data
drift
+
+## RAG (Retrieval-Augmented Generation)
+
+RAG retrieves relevant information from external knowledge bases to provide
context for large models, effectively addressing model hallucination and
knowledge currency issues. The vector engine is a core component of RAG
systems, requiring rapid recall of the most relevant document fragments from
massive knowledge bases while supporting high-concurrency user query requests
to ensure application responsiveness.
+
+- **Enterprise knowledge**: Building intelligent Q&A systems based on internal
documents and manuals, enabling employees to quickly obtain accurate answers
through natural language
+- **Intelligent customer service assistant**: Combining product knowledge
bases and historical cases to provide precise response suggestions for customer
service personnel or chatbots
+- **Intelligent document assistant**: Rapidly locating relevant content in
large-scale document collections to assist research, writing, and
decision-making processes
+
+Building RAG applications based on Doris offers the following advantages in
these scenarios:
+
+- **High concurrency performance**: Distributed architecture supports
high-concurrency vector retrieval, easily handling large-scale concurrent user
access
+- **Hybrid retrieval capability**: Single SQL executes vector similarity
search and keyword filtering simultaneously, balancing semantic recall and
exact matching
+- **Elastic scaling**: Query performance scales linearly with cluster
expansion, seamlessly transitioning from millions to tens of billions of vectors
+- **Unified solution**: Uniformly manages vector data, original documents, and
business data, simplifying the data architecture for RAG applications
+
+## AI Observability
+
+AI model training iterations and application operations generate massive
amounts of logs, metrics, and tracing data. To precisely locate issues and
continuously optimize performance, observability systems have become a critical
component of AI infrastructure. As business scale expands, observability
platforms face multiple challenges including high-throughput writes of PB-scale
data, millisecond-level retrieval response, and cost control. Typical use cases
include:
+
+- **Model training monitoring**: Real-time tracking of training metrics and
resource consumption, rapidly identifying training anomalies and performance
bottlenecks
+- **Inference service tracing**: Recording the complete trace of each
inference request, analyzing latency sources and error patterns
+- **AI** **application log analysis**: Full-text retrieval and aggregation
analysis of massive application logs, supporting troubleshooting and behavioral
insights
+
+Building AI Observability with Doris offers the following advantages:
+
+- **Ultimate performance**: Supports sustained writes of PB/day (10GB/s),
inverted indexes accelerate log retrieval with second-level response
+- **Cost optimization**: Compression ratios of 5:1 to 10:1, storage cost
savings of 50%-80%, supports low-cost storage for cold data
+- **Flexible schema**: Light Schema Change enables second-level field
modifications, VARIANT type natively supports dynamic JSON structures
+- **Ecosystem-friendly**: Compatible with OpenTelemetry and ELK ecosystems,
supports integration with Grafana/Kibana visualization tools
+
+## Semantic Search
+
+Semantic search captures the deep meaning of text through vectorization
techniques. Even when query terms differ from document wording, semantically
relevant content can still be retrieved. This is crucial for scenarios such as
cross-language retrieval, synonym recognition, and intent understanding,
significantly improving search recall rates and user experience. Typical use
cases include:
+
+- **Enterprise document retrieval**: Employees describe issues in natural
language, and the system understands intent to recall semantically relevant
policies, procedures, and knowledge from massive documents
+- **E-commerce product search**: Users input "breathable shoes suitable for
summer," and the system understands the need to recall relevant products rather
than merely matching keywords
+- **Content recommendation**: Intelligent recommendations based on semantic
similarity of articles and videos, discovering content of potential interest
with different wording
+
+Building semantic search applications based on Doris offers the following
advantages:
+
+- **High-performance vector retrieval**: Supports HNSW and IVF algorithms,
sub-second response for hundred-million-scale vectors, easily handling
large-scale semantic search requirements
+- **Enhanced hybrid retrieval**: Single SQL integrates semantic search and
keyword filtering, ensuring necessary vocabulary hits while recalling
semantically relevant content
+- **Multimodal extension**: Supports not only text semantic search but can
also extend to semantic retrieval of multimodal content such as images and audio
+- **Flexible quantization optimization**: Through SQ/PQ quantization
techniques, significantly reduces storage and computing costs while maintaining
retrieval accuracy
diff --git a/versioned_sidebars/version-4.x-sidebars.json
b/versioned_sidebars/version-4.x-sidebars.json
index 1b1ac274955..8a974cf715a 100644
--- a/versioned_sidebars/version-4.x-sidebars.json
+++ b/versioned_sidebars/version-4.x-sidebars.json
@@ -401,6 +401,7 @@
"type": "category",
"label": "AI",
"items": [
+ "ai/ai-overview",
"ai/ai-function-overview",
{
"type": "category",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
