hgaol commented on issue #1468:
URL: https://github.com/apache/answer/issues/1468#issuecomment-3723067855
**Share arch design below for better discussion.**
## Overview
The Copilot Chat plugin provides an AI-powered chat assistant with RAG
(Retrieval-Augmented Generation) capabilities for Apache Answer. It integrates
with LLM providers for chat and embeddings, with support for multiple vector
store backends.
### Current Support
| Component | Supported Options |
|-----------|-------------------|
| **LLM Provider** | Azure OpenAI (gpt-4, gpt-4o, etc.) |
| **Embedding Provider** | Uses selected LLM provider's embedding API (Azure
OpenAI: text-embedding-ada-002, text-embedding-3-small, etc.) |
| **Vector Store** | In-Memory, Qdrant |
> **Note:** Currently, the embedding provider is automatically set to match
the LLM provider. Provider-specific embedding settings are shown based on the
selected LLM provider.
### Future Extensibility
The architecture is designed to be extensible. Future versions could support:
- **Additional LLM Providers**: OpenAI (direct), Anthropic Claude, Google
Gemini, local models (Ollama)
- **Additional Embedding Providers**: OpenAI, Cohere, HuggingFace, local
embeddings
- **Additional Vector Stores**: Pinecone, Weaviate, Milvus, PostgreSQL
(pgvector), Chroma
## Plugin Structure
```
copilot-chat/
├── copilot_chat.go # Backend: Go plugin implementation
├── Component.tsx # Frontend: React chat UI component
├── index.ts # Frontend: Plugin entry point
├── info.yaml # Plugin metadata (slug_name: copilot_chat, type:
agent)
├── go.mod / go.sum # Go dependencies
├── package.json # Frontend dependencies
├── i18n/ # Internationalization
│ ├── en_US.yaml
│ ├── zh_CN.yaml
│ ├── index.ts
│ └── translation.go
└── vite.config.ts # Frontend build config
```
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────────┐
│ Apache Answer Core │
├─────────────────────────────────────────────────────────────────────┤
│ plugin_common_service.go │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ UpdatePluginConfig() │ │
│ │ - Detects copilot_chat plugin │ │
│ │ - Calls agent.SetSyncer(SearchSyncer) for RAG access │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ search_sync.go → NewPluginSyncer() │ │
│ │ - GetQuestionsPage() │ │
│ │ - GetAnswersPage() │ │
│ └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ copilot-chat Plugin │
├─────────────────────────────────────────────────────────────────────┤
│ Backend (copilot_chat.go) │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ CopilotChatPlugin implements plugin.Agent │ │
│ │ - ConfigFields() / ConfigReceiver() │ │
│ │ - RegisterAuthUserRouter() → /copilot-chat/chat (SSE) │ │
│ │ - SetSyncer(SearchSyncer) │ │
│ │ - UpdateEmbeddings() (cron job) │ │
│ │ - SimilaritySearch() │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ Vector Stores │
│ ┌──────────────────────┐ ┌──────────────────────┐ │
│ │ MemoryVectorStore │ │ QdrantVectorStore │ │
│ │ (in-memory, dev) │ │ (persistent, prod) │ │
│ └──────────────────────┘ └──────────────────────┘ │
│ │
│ External Services │
│ ┌──────────────────────┐ ┌──────────────────────┐ │
│ │ Azure OpenAI (LLM) │ │ Azure OpenAI (Embed) │ │
│ │ - gpt-4 │ │ - text-embedding-* │ │
│ └──────────────────────┘ └──────────────────────┘ │
├─────────────────────────────────────────────────────────────────────┤
│ Frontend (Component.tsx) │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ - Floating/Docked chat panel │ │
│ │ - Markdown rendering (marked.js) │ │
│ │ - Chat history persistence (localStorage) │ │
│ │ - Reference links display │ │
│ └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Key Components
### Backend (Go)
| Component | Description |
|-----------|-------------|
| `CopilotChatPlugin` | Main plugin struct implementing `plugin.Agent`
interface |
| `SetSyncer()` | Receives `SearchSyncer` from Answer core for Q&A data
access |
| `UpdateEmbeddings()` | Fetches all Q&A content and creates vector
embeddings |
| `SimilaritySearch()` | Searches vector store for relevant context |
| `handleChat()` | POST endpoint returning full JSON response |
| `MemoryVectorStore` | In-memory vector store (no external deps) |
| `QdrantVectorStoreWrapper` | Qdrant integration with deduplication |
### Frontend (React/TypeScript)
| Component | Description |
|-----------|-------------|
| `Component.tsx` | Main chat UI with floating/docked modes |
| `localStorage` | Persists chat history, panel state, width |
## Changes Required in Answer Core
The following changes are needed in the `apache/answer` repository:
### 1. Plugin Service Integration
**File:** `internal/service/plugin_common/plugin_common_service.go`
When a plugin config is saved, automatically detect any Agent plugin that
implements `SetSyncer(plugin.SearchSyncer)` and inject the syncer. This allows
Agent plugins to access Q&A content for RAG without hardcoding plugin names.
### 2. Plugin Config Field Dependencies (DependsOn/ShowWhen)
Support for conditional field visibility based on other field values.
**Files:**
- `plugin/config.go` - Add `DependsOn` and `ShowWhen` fields to
`ConfigFieldUIOptions` struct
- `internal/schema/plugin_admin_schema.go` - Add same fields to schema for
API response
- `ui/src/pages/Admin/Plugins/Config/index.tsx` - Frontend logic to:
- Read `depends_on` and `show_when` from backend config
- Apply visibility rules when form values change
- Hide fields when parent field value doesn't match `show_when` values
## Configuration Options
### LLM Provider Selection
| Option | Description |
|--------|-------------|
| `llm_provider` | LLM provider to use (currently: Azure OpenAI) |
### Azure OpenAI Settings
*Shown when `llm_provider` = azure_openai*
| Option | Description | Depends On |
|--------|-------------|------------|
| `azure_endpoint` | Azure OpenAI endpoint URL | `llm_provider` =
azure_openai |
| `azure_api_key` | Azure OpenAI API key | `llm_provider` = azure_openai |
| `azure_deployment` | LLM deployment name (e.g., gpt-4) | `llm_provider` =
azure_openai |
| `azure_api_version` | API version (e.g., 2024-02-15-preview) |
`llm_provider` = azure_openai |
### Chat Settings
| Option | Description |
|--------|-------------|
| `system_prompt` | Custom system prompt |
| `max_tokens` | Max response tokens |
| `temperature` | Response randomness (0.0-1.0) |
| `require_login` | Require user login to access chat |
| `enable_rag` | Enable RAG with Q&A context |
### RAG Settings
*Shown when `enable_rag` = true*
| Option | Description | Depends On |
|--------|-------------|------------|
| `chunk_level` | `item` (per answer) or `question` (full thread) |
`enable_rag` = true |
| `top_k` | Number of context documents | `enable_rag` = true |
| `similarity_threshold` | Min similarity score (0.0-1.0) | `enable_rag` =
true |
| `embedding_cron` | Cron schedule for embedding updates | `enable_rag` =
true |
| `vector_store_type` | `memory` or `qdrant` | `enable_rag` = true |
### Embedding Provider Settings
> **Note:** Currently, the embedding provider uses the same provider as the
LLM. The embedding deployment config is shown when RAG is enabled. Future
versions may support independent embedding provider selection.
#### Azure OpenAI Embedding
*Shown when `enable_rag` = true*
| Option | Description | Depends On |
|--------|-------------|------------|
| `azure_embedding_deployment` | Embedding model deployment (e.g.,
text-embedding-ada-002) | `enable_rag` = true |
### Qdrant Settings
*Shown when `vector_store_type` = qdrant*
| Option | Description | Depends On |
|--------|-------------|------------|
| `qdrant_url` | Qdrant server URL | `vector_store_type` = qdrant |
| `qdrant_api_key` | Qdrant API key (optional) | `vector_store_type` =
qdrant |
| `qdrant_collection_name` | Qdrant collection name | `vector_store_type` =
qdrant |
## API Endpoints
| Method | Endpoint | Auth | Description |
|--------|----------|------|-------------|
| POST | `/answer/api/v1/copilot-chat/chat` | User | Chat (JSON response) |
| GET | `/answer/api/v1/copilot-chat/status` | Admin | Plugin status |
| POST | `/answer/api/v1/copilot-chat/embeddings/update` | Admin | Trigger
embedding update |
| GET | `/answer/api/v1/copilot-chat/embeddings/status` | Admin | Embedding
status |
## Data Flow (RAG)
1. **Embedding Phase** (cron or manual trigger):
- Plugin calls `syncer.GetQuestionsPage()` / `syncer.GetAnswersPage()`
- Content is chunked based on `chunk_level` setting
- Azure OpenAI creates embeddings
- Embeddings stored in vector store
2. **Query Phase**:
- User sends POST request with message
- Plugin performs similarity search
- Top-K relevant documents retrieved
- Context + user message sent to Azure OpenAI LLM
- Full JSON response returned to frontend
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
