This is an automated email from the ASF dual-hosted git repository.
moonming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/apisix.git
The following commit(s) were added to refs/heads/master by this push:
new 2228b64aa5 docs(ai): port current protocol and streaming guidance
(#13731)
2228b64aa5 is described below
commit 2228b64aa572d92e75fbc7b13a1d421ccd22b525
Author: Traky Deng <[email protected]>
AuthorDate: Tue Nov 10 16:26:19 2026 +0800
docs(ai): port current protocol and streaming guidance (#13731)
---
.../latest/plugins/ai-aliyun-content-moderation.md | 6 +-
.../en/latest/plugins/ai-aws-content-moderation.md | 2 +-
docs/en/latest/plugins/ai-cache.md | 2 +
docs/en/latest/plugins/ai-lakera-guard.md | 6 +-
docs/en/latest/plugins/ai-prompt-decorator.md | 4 +
docs/en/latest/plugins/ai-prompt-guard.md | 4 +
docs/en/latest/plugins/ai-prompt-template.md | 113 ++++++++++++++++++++-
docs/en/latest/plugins/ai-proxy-multi.md | 82 +++++++++++++--
docs/en/latest/plugins/ai-proxy.md | 82 +++++++++++++--
docs/en/latest/plugins/ai-rag.md | 4 +
docs/en/latest/plugins/ai-request-rewrite.md | 4 +-
.../latest/plugins/ai-aliyun-content-moderation.md | 6 +-
.../zh/latest/plugins/ai-aws-content-moderation.md | 2 +-
docs/zh/latest/plugins/ai-cache.md | 2 +
docs/zh/latest/plugins/ai-lakera-guard.md | 6 +-
docs/zh/latest/plugins/ai-prompt-decorator.md | 4 +
docs/zh/latest/plugins/ai-prompt-guard.md | 4 +
docs/zh/latest/plugins/ai-prompt-template.md | 113 ++++++++++++++++++++-
docs/zh/latest/plugins/ai-proxy-multi.md | 82 +++++++++++++--
docs/zh/latest/plugins/ai-proxy.md | 86 +++++++++++++---
docs/zh/latest/plugins/ai-rag.md | 4 +
docs/zh/latest/plugins/ai-request-rewrite.md | 4 +-
22 files changed, 566 insertions(+), 56 deletions(-)
diff --git a/docs/en/latest/plugins/ai-aliyun-content-moderation.md
b/docs/en/latest/plugins/ai-aliyun-content-moderation.md
index b2e5483115..6a9673ef22 100644
--- a/docs/en/latest/plugins/ai-aliyun-content-moderation.md
+++ b/docs/en/latest/plugins/ai-aliyun-content-moderation.md
@@ -45,6 +45,10 @@ Please ensure that the `access_key_secret` is correctly
configured in the Plugin
The `ai-aliyun-content-moderation` Plugin should be used with either
[`ai-proxy`](./ai-proxy.md) or [`ai-proxy-multi`](./ai-proxy-multi.md) Plugin
for proxying LLM requests.
+The Plugin moderates Chat Completions, Responses API, Embeddings, Anthropic
Messages, and Bedrock Converse requests using each protocol's native content
structure. For Responses, it moderates `input` and returns blocked
non-streaming or streaming requests in Responses API format. For Embeddings, it
moderates a string or an array of strings in `input` and returns rejected
requests as OpenAI-style errors. Other non-empty JSON bodies use passthrough,
from which the Plugin extracts no format [...]
+
+See [request protocol detection](./ai-proxy.md#request-protocol-detection) for
the detection order and URI requirements.
+
## Attributes
| Name | Type | Required | Default | Valid values | Description |
@@ -65,7 +69,7 @@ The `ai-aliyun-content-moderation` Plugin should be used with
either [`ai-proxy`
| response_check_service | string | False | `"llm_response_moderation"` | |
Aliyun service for response moderation. |
| response_check_length_limit | number | False | `5000` | >= 1 | Response
content length limit. If exceeded, the content is sent to Aliyun in chunks. For
instance, if the response content is 250 characters and
`response_check_length_limit` is set to `100`, the content is sent in 3
requests to Aliyun. |
| risk_level_bar | string | False | `"high"` | `none`, `low`, `medium`,
`high`, `max` | If the evaluated risk level is lower than the `risk_level_bar`,
the request or response will be passed through to Upstream LLM or client
respectively. |
-| deny_code | integer | False | `200` | [200, 599] | Rejection HTTP status
code. Defaults to `200` so the provider-compatible refusal parses as a normal
completion in client SDKs; set a 4xx to surface denies as HTTP errors instead. |
+| deny_code | integer | False | `200` | [200, 599] | Rejection HTTP status
code. The default `200` returns the denial in the detected protocol's response
shape. Set a 4xx value to surface denials as HTTP errors instead. |
| deny_message | string | False | | | Rejection message. |
| timeout | integer | False | `10000` | >= 1 | Timeout in milliseconds. |
| keepalive | boolean | False | `true` | | If `true`, enable HTTP connection
keepalive to Aliyun. |
diff --git a/docs/en/latest/plugins/ai-aws-content-moderation.md
b/docs/en/latest/plugins/ai-aws-content-moderation.md
index f44632c94c..5c47edbcb9 100644
--- a/docs/en/latest/plugins/ai-aws-content-moderation.md
+++ b/docs/en/latest/plugins/ai-aws-content-moderation.md
@@ -38,7 +38,7 @@ import TabItem from '@theme/TabItem';
The `ai-aws-content-moderation` Plugin integrates with [AWS
Comprehend](https://aws.amazon.com/comprehend/) to check content for toxicity
when proxying to LLMs, such as profanity, hate speech, insult, harassment,
violence, and more, rejecting requests if the evaluated outcome exceeds the
configured threshold.
-The Plugin is protocol-aware: it extracts the prompt content from the LLM
request (for example `messages[].content`) and moderates only that decoded
text, rather than the raw request body. `request_check_roles` selects which
message roles are moderated, and `request_check_mode` can narrow `user`/`tool`
moderation to the newest turn so conversation history is not re-scored on every
request.
+The Plugin supports Chat Completions, Responses API, Embeddings, Anthropic
Messages, and Bedrock Converse requests. It extracts text using the detected
protocol's native content structure and moderates only that decoded text rather
than the raw request body. Rejected requests use the detected protocol's
response format. `request_check_roles` selects which message roles are
moderated, and `request_check_mode` can narrow `user`/`tool`/`assistant`
moderation to the newest turn so conversati [...]
Both directions can be moderated. Set `check_response` to moderate the LLM
response as well. For streaming responses, `stream_check_mode` selects between
`realtime`, which moderates batches as they arrive and replaces the remainder
of the stream once a batch is flagged, and `final_packet`, which moderates the
assembled response and annotates the last chunk with `risk_level`. The verdict
is also exposed on the request context as `$llm_content_risk_level` (`high` or
`none`) for logging.
diff --git a/docs/en/latest/plugins/ai-cache.md
b/docs/en/latest/plugins/ai-cache.md
index 6018ddf1e7..a7b4813677 100644
--- a/docs/en/latest/plugins/ai-cache.md
+++ b/docs/en/latest/plugins/ai-cache.md
@@ -45,6 +45,8 @@ This Plugin supports two cache layers:
- **Exact (L1):** A SHA-256 fingerprint of the effective prompt is used as the
Redis key. An identical prompt always hits the same entry.
- **Semantic (L2):** When L1 misses, the prompt is embedded into a vector and
a nearest-neighbour search retrieves a past response whose embedding is within
the configured similarity threshold. L2 is disabled by default; enable it by
adding `"semantic"` to `layers`.
+Exact caching supports Chat Completions, Responses API, Embeddings, Anthropic
Messages, Bedrock Converse, and passthrough requests and keeps the protocols in
separate cache entries. For passthrough, the cache key also includes the
request method, URI, and query string. Semantic caching applies only to Chat
Completions requests; the other protocols can use the exact layer but bypass
the semantic layer.
+
The `ai-cache` Plugin must be used with the [`ai-proxy`](./ai-proxy.md) or
[`ai-proxy-multi`](./ai-proxy-multi.md) Plugin.
### Streaming
diff --git a/docs/en/latest/plugins/ai-lakera-guard.md
b/docs/en/latest/plugins/ai-lakera-guard.md
index c43a432aa6..9eb071fd9f 100644
--- a/docs/en/latest/plugins/ai-lakera-guard.md
+++ b/docs/en/latest/plugins/ai-lakera-guard.md
@@ -43,7 +43,9 @@ The `ai-lakera-guard` Plugin integrates with the [Lakera
Guard API (v2)](https:/
Which detectors run and at what thresholds are controlled entirely by the
**Lakera project policy**, selected with `project_id`. There is no gateway-side
detector list; Lakera returns a single verdict per call.
-The `ai-lakera-guard` Plugin should be used with either the
[`ai-proxy`](./ai-proxy.md) or [`ai-proxy-multi`](./ai-proxy-multi.md) Plugin
for proxying LLM requests. It relies on the context that `ai-proxy` populates
to extract chat content in a protocol-aware way.
+The `ai-lakera-guard` Plugin should be used with either the
[`ai-proxy`](./ai-proxy.md) or [`ai-proxy-multi`](./ai-proxy-multi.md) Plugin
for proxying LLM requests. It relies on the context that `ai-proxy` populates
to extract request content in a protocol-aware way.
+
+Request scanning supports Chat Completions, Responses API, Embeddings,
Anthropic Messages, and Bedrock Converse requests. For Responses, the Plugin
converts `instructions` and text from `input` into conversation messages. For
Embeddings, it scans text from `input`. Response scanning applies to the
protocols that return generated text; Embeddings responses contain vectors
rather than text. Passthrough requests and responses are not scanned.
Requests that did not pass through `ai-proxy`/`ai-proxy-multi` (for example
plain HTTP traffic when the Plugin is bound at the Consumer or Service level)
cannot be inspected. By default such requests are passed through unchecked;
this is configurable via `fail_mode`.
@@ -63,7 +65,7 @@ The Plugin can scan the request prompt (`direction: input`),
the LLM response (`
| timeout | integer | False | `5000` | >= 1 | Lakera request timeout in
milliseconds. |
| ssl_verify | boolean | False | `true` | | If `true`, verify the TLS
certificate of the Lakera endpoint. |
| reveal_failure_categories | boolean | False | `false` | | If `true`, append
the matched Lakera `detector_type`s (with their confidence result) to the deny
message returned to the client. The full per-detector `breakdown` is always
requested from Lakera and written to the gateway logs regardless of this
setting; this flag only controls client-facing exposure. |
-| deny_code | integer | False | `200` | 200 - 599 | HTTP status code returned
when a request is blocked. Defaults to `200` so the body — a
provider-compatible chat completion (or SSE) carrying `request_failure_message`
— parses as a normal refusal in client SDKs (matching how Lakera Guard itself
returns `200` with a verdict). Set a 4xx (e.g. `403`) if you prefer blocks to
surface as HTTP errors. |
+| deny_code | integer | False | `200` | 200 - 599 | HTTP status code returned
when traffic is blocked. The default `200` returns the denial in the detected
protocol's response shape with `request_failure_message` or
`response_failure_message`. Set a 4xx value, such as `403`, to surface denials
as HTTP errors instead. |
| request_failure_message | string | False | `Request blocked by Lakera Guard`
| | Refusal text returned (as the assistant message of a provider-compatible
response) when a request is blocked. |
| response_failure_message | string | False | `Response blocked by Lakera
Guard` | | Refusal text returned (as the assistant message of a
provider-compatible response) when an LLM response is blocked (`direction`
`output` or `both`). |
diff --git a/docs/en/latest/plugins/ai-prompt-decorator.md
b/docs/en/latest/plugins/ai-prompt-decorator.md
index ae351dd11d..0c4cb6cccc 100644
--- a/docs/en/latest/plugins/ai-prompt-decorator.md
+++ b/docs/en/latest/plugins/ai-prompt-decorator.md
@@ -38,6 +38,10 @@ import TabItem from '@theme/TabItem';
The `ai-prompt-decorator` Plugin modifies user input prompts by prefixing and
appending pre-engineered prompts to set contexts in content generation. This
practice helps the model operate within desired guidelines during interactions.
+The Plugin decorates Chat Completions, Responses API, Anthropic Messages, and
Bedrock Converse requests using each protocol's native prompt structure. For
Responses, content from `prepend` is added to `instructions`, and content from
`append` is added to `input`. The Plugin leaves Embeddings and passthrough
requests unchanged because they have no prompt roles to prepend or append.
+
+See [request protocol detection](./ai-proxy.md#request-protocol-detection) for
the detection order and URI requirements.
+
## Plugin Attributes
| Name | Type | Required | Default | Valid values | Description |
diff --git a/docs/en/latest/plugins/ai-prompt-guard.md
b/docs/en/latest/plugins/ai-prompt-guard.md
index 7cf96a0910..7c621d4763 100644
--- a/docs/en/latest/plugins/ai-prompt-guard.md
+++ b/docs/en/latest/plugins/ai-prompt-guard.md
@@ -40,6 +40,10 @@ The `ai-prompt-guard` Plugin safeguards your LLM endpoints
by inspecting and val
When both `allow_patterns` and `deny_patterns` are configured, the Plugin
first ensures that at least one `allow_patterns` is matched. If none match, the
request is rejected. If an allowed pattern is matched, it then checks for any
occurrences of denied patterns.
+The Plugin checks Chat Completions, Responses API, Embeddings, Anthropic
Messages, and Bedrock Converse requests using each protocol's native content
structure. For Responses, it checks user content in `input` and, when
`match_all_roles` is `true`, system content in `instructions`.
+
+See [request protocol detection](./ai-proxy.md#request-protocol-detection) for
the detection order and URI requirements. Other non-empty JSON bodies use
passthrough and are handled according to `fail_mode` rather than inspected as a
supported AI request.
+
For Embeddings requests, the Plugin checks a string `input` or all strings in
an `input` array. Batch items are joined with spaces and checked together,
regardless of `match_all_conversation_history`. Allow patterns apply to this
combined text; a denied pattern anywhere in the batch rejects the request.
When no content is selected for inspection, the request continues upstream.
This includes messages excluded by role filtering and Embeddings inputs
containing only token IDs, which the Plugin does not decode into text.
`fail_mode: "error"` applies to unrecognized protocols; it does not reject
recognized requests solely because no text was selected.
diff --git a/docs/en/latest/plugins/ai-prompt-template.md
b/docs/en/latest/plugins/ai-prompt-template.md
index b176aec944..82d1e6b60b 100644
--- a/docs/en/latest/plugins/ai-prompt-template.md
+++ b/docs/en/latest/plugins/ai-prompt-template.md
@@ -38,6 +38,8 @@ import TabItem from '@theme/TabItem';
The `ai-prompt-template` Plugin supports pre-configuring prompt templates that
only accept user inputs in designated template variables, in a "fill in the
blank" fashion. It simplifies access to LLM providers, such as OpenAI and
Anthropic, by letting you define reusable prompt structures.
+The Plugin replaces the incoming request body with the selected rendered JSON
template. A template can use any request format supported by the downstream
Plugin, including Chat Completions, Responses API, and Embeddings. Use
[`ai-prompt-decorator`](./ai-prompt-decorator.md) instead when you need to add
content to an existing request without replacing its body.
+
## Plugin Attributes
| Name | Type | Required | Default | Valid values | Description |
@@ -45,7 +47,7 @@ The `ai-prompt-template` Plugin supports pre-configuring
prompt templates that o
| `max_req_body_size` | integer | False | 67108864 | >= 1 | Maximum request
body size in bytes buffered into memory. Requests with a larger body are
rejected. |
| `templates` | array | True | | | An array of template objects. |
| `templates.name` | string | True | | | Name of the template. When requesting
the Route, the request should include the template name that corresponds to the
configured template. |
-| `templates.template` | object | True | | | Template specification. |
+| `templates.template` | object | True | | | JSON request body template. It
can contain any fields accepted by the downstream Plugin. |
| `templates.template.model` | string | False | | | Name of the LLM model,
such as `gpt-4` or `gpt-3.5`. See your LLM provider API documentation for more
available models. |
| `templates.template.messages` | array[object] | False | | | Template message
specification. |
| `templates.template.messages.role` | string | True | | [`system`, `user`,
`assistant`] | Role of the message. |
@@ -601,3 +603,112 @@ You should receive a response similar to the following:
...
}
```
+
+### Configure a Responses API Web Search Template
+
+The following example configures a Responses API template that searches only
the official Apache APISIX website. The Route URI can have a custom prefix, but
it must end in `/v1/responses` so that `ai-proxy` distinguishes the rendered
body from an Embeddings request.
+
+Create a Route with the `ai-proxy` and `ai-prompt-template` Plugins:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/ai-prompt-template-responses";
-X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/template/v1/responses",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-proxy": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ }
+ },
+ "ai-prompt-template": {
+ "templates": [
+ {
+ "name": "Search APISIX documentation",
+ "template": {
+ "model": "gpt-4.1",
+ "tools": [
+ {
+ "type": "web_search",
+ "filters": {
+ "allowed_domains": ["apisix.apache.org"]
+ }
+ }
+ ],
+ "tool_choice": "required",
+ "input": "Search the official Apache APISIX website for
{{topic}} and summarize the result in one sentence."
+ }
+ }
+ ]
+ }
+ }
+ }'
+```
+
+Send the template name and a value for `topic` to the Route:
+
+```shell
+curl "http://127.0.0.1:9080/template/v1/responses"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "Search APISIX documentation",
+ "topic": "what APISIX is"
+ }'
+```
+
+Before `ai-proxy` processes the request, `ai-prompt-template` replaces the
incoming body with the rendered template:
+
+```json
+{
+ "model": "gpt-4.1",
+ "tools": [
+ {
+ "type": "web_search",
+ "filters": {
+ "allowed_domains": ["apisix.apache.org"]
+ }
+ }
+ ],
+ "tool_choice": "required",
+ "input": "Search the official Apache APISIX website for what APISIX is and
summarize the result in one sentence."
+}
+```
+
+APISIX identifies the rendered request as Responses API from its `input` field
and the Route URI suffix. You should receive an HTTP `200` response whose
`output` contains a completed web search call and a message with URL citations.
The generated text, search details, and citation offsets vary by request; the
response shape is similar to the following:
+
+```json
+{
+ "status": "completed",
+ "output": [
+ {
+ "type": "web_search_call",
+ "status": "completed",
+ "action": {
+ "type": "search"
+ }
+ },
+ {
+ "type": "message",
+ "status": "completed",
+ "role": "assistant",
+ "content": [
+ {
+ "type": "output_text",
+ "text": "<generated summary with a citation>",
+ "annotations": [
+ {
+ "type": "url_citation",
+ "title": "Apache APISIX",
+ "url": "https://apisix.apache.org/";
+ }
+ ]
+ }
+ ]
+ }
+ ]
+}
+```
diff --git a/docs/en/latest/plugins/ai-proxy-multi.md
b/docs/en/latest/plugins/ai-proxy-multi.md
index 2246fe8aa6..ddc5472b3b 100644
--- a/docs/en/latest/plugins/ai-proxy-multi.md
+++ b/docs/en/latest/plugins/ai-proxy-multi.md
@@ -44,6 +44,12 @@ In addition, the Plugin also supports logging LLM request
information in the acc
## Request Format
+### Request Protocol Detection
+
+`ai-proxy-multi` uses the same request validation, protocol detection,
passthrough, and conversion behavior as `ai-proxy`. See [request protocol
detection](./ai-proxy.md#request-protocol-detection) for the complete rules and
URI requirements.
+
+### Chat Completions Request Format
+
| Name | Type | Required | Description
|
| ------------------ | ------ | -------- |
--------------------------------------------------- |
| `messages` | Array | True | An array of message objects.
|
@@ -77,7 +83,7 @@ When an instance's `provider` is set to `bedrock`, the Plugin
expects requests i
| balancer.key | string | False |
| | Used when `type` is `chash`. When
`hash_on` is set to `header` or `cookie`, `key` is required. When `hash_on` is
set to `consumer`, `key` is not required as the consumer name will be used as
the key automatically. |
| instances | array[object] | True |
| | LLM instance configurations. |
| instances.name | string | True |
| | Name of the LLM service instance. It
must be unique within `instances`, since it identifies the instance in the
balancer, in its health checker, and in other Plugins that reference it, such
as `ai-rate-limiting`. |
-| instances.provider | string | True |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM service
provider. When set to `openai`, the Plugin will proxy the request to
`api.openai.com`. When set to `deepseek`, the Plugin will proxy the request to
`api.deepseek.com`. When set to `aimlapi`, the Plugin uses the
OpenAI-compatible driver and proxies the request to [...]
+| instances.provider | string | True |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM service
provider. When set to `openai`, the Plugin sends detected Chat Completions,
Responses, and Embeddings requests to the corresponding endpoint on
`api.openai.com`. When set to `deepseek`, the Plugin will proxy the request to
`api.deepseek.com`. When set to `aimlapi [...]
| instances.provider_conf | object | False |
| | Configuration for the specific
provider. Required when `provider` is set to `vertex-ai` and `override` is not
configured. Required when `provider` is set to `bedrock`. |
| instances.provider_conf.project_id | string | True |
| | Google Cloud Project ID. |
| instances.provider_conf.region | string | True (depending on
provider) | | minLength = 1 (for Bedrock) |
When `provider` is `vertex-ai`, this is the Google Cloud Region. When
`provider` is `bedrock`, this is the AWS region used to construct the Bedrock
endpoint and to sign the request with SigV4 (required, must be non-empty). |
@@ -145,6 +151,7 @@ When an instance's `provider` is set to `bedrock`, the
Plugin expects requests i
| keepalive | boolean | False | true
| | If true, keep the connection alive when
requesting the LLM service. |
| keepalive_timeout | integer | False | 60000
| greater than or equal to 1000 | Request timeout in
milliseconds when requesting the LLM service. |
| keepalive_pool | integer | False | 30
| | Keepalive pool size for when connecting
with the LLM service. |
+| streaming_flush_interval_ms | integer | False | 10
| greater than or equal to 0 | Background flush interval
in milliseconds for streaming responses. A positive value periodically flushes
buffered output to bound client latency when the upstream sends tokens in
bursts. Set to `0` to flush each chunk synchronously. |
| ssl_verify | boolean | False | true
| | If true, verify the LLM service's
certificate. |
## Request Header Forwarding
@@ -2086,6 +2093,70 @@ If `auth.aws.session_token` is set, it is used for
temporary credentials (e.g.,
To enable streaming, send the same Converse request body with `"stream":
true`. The Plugin routes the request to Bedrock's
`/model/<model>/converse-stream` endpoint and forwards each AWS EventStream
frame to the client unchanged. The response `Content-Type` is
`application/vnd.amazon.eventstream`; clients must parse the binary framing
themselves (most AWS SDKs do this automatically).
+### Load Balance Responses API Requests
+
+The following example configures `ai-proxy-multi` to distribute Responses API
requests across two OpenAI models. Responses and Embeddings requests both
contain `input`, so the Route URI must end in `/v1/responses` for the Plugin to
identify the request as Responses.
+
+Obtain an [OpenAI API key](https://platform.openai.com/api-keys) and save it
to an environment variable:
+
+```shell
+export OPENAI_API_KEY="<your-api-key>"
+```
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "ai-proxy-multi-responses-route",
+ "uri": "/v1/responses",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-proxy-multi": {
+ "instances": [
+ {
+ "name": "openai-responses-primary",
+ "provider": "openai",
+ "weight": 1,
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4.1"
+ }
+ },
+ {
+ "name": "openai-responses-secondary",
+ "provider": "openai",
+ "weight": 1,
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4.1-mini"
+ }
+ }
+ ]
+ }
+ }
+ }'
+```
+
+Send a request using the Responses API format:
+
+```shell
+curl "http://127.0.0.1:9080/v1/responses"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "input": "Explain API gateways in one sentence."
+ }'
+```
+
+The request is forwarded to one of the configured OpenAI instances and the
response is returned in the Responses API format.
+
### Proxy to Embedding Models
The following example demonstrates how you can configure the `ai-proxy-multi`
Plugin to proxy requests and load balance between embedding models.
@@ -2124,9 +2195,6 @@ curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
},
"options": {
"model": "text-embedding-3-small"
- },
- "override": {
- "endpoint": "https://api.openai.com/v1/embeddings";
}
},
{
@@ -2175,8 +2243,6 @@ services:
Authorization: "Bearer ${OPENAI_API_KEY}"
options:
model: text-embedding-3-small
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
- name: az-openai-instance
provider: azure-openai
weight: 0
@@ -2228,8 +2294,6 @@ spec:
Authorization: "Bearer your-api-key"
options:
model: text-embedding-3-small
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
- name: az-openai-instance
provider: azure-openai
weight: 0
@@ -2295,8 +2359,6 @@ spec:
Authorization: "Bearer your-api-key"
options:
model: text-embedding-3-small
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
- name: az-openai-instance
provider: azure-openai
weight: 0
diff --git a/docs/en/latest/plugins/ai-proxy.md
b/docs/en/latest/plugins/ai-proxy.md
index 090d192548..3f01f84244 100644
--- a/docs/en/latest/plugins/ai-proxy.md
+++ b/docs/en/latest/plugins/ai-proxy.md
@@ -44,6 +44,28 @@ In addition, the Plugin also supports logging LLM request
information in the acc
## Request Format
+### Request Protocol Detection
+
+The Plugin detects the client request protocol before selecting a compatible
upstream endpoint. It checks the following rules in order:
+
+| Client protocol | Detection | Route URI |
+| --- | --- | --- |
+| Bedrock Converse | The request body contains a `messages` array and the
request URI ends in `/converse`. | The URI can have a custom prefix, but it
must keep the `/converse` suffix. |
+| Anthropic Messages | The request body is a JSON object or array and the
request URI ends in `/v1/messages`. | The URI can have a custom prefix, but it
must keep the `/v1/messages` suffix. |
+| OpenAI Responses | The request body contains `input` and the request URI
ends in `/v1/responses`. | The URI can have a custom prefix, but it must keep
the `/v1/responses` suffix. |
+| OpenAI Chat Completions | The request body contains a `messages` array. |
Any URI matched by the Route. |
+| OpenAI Embeddings | The request body contains `input`, and no earlier rule
matched. | Any URI matched by the Route. |
+
+The URI-specific rules run before the body-only rules. This prevents Bedrock
Converse and Anthropic Messages requests containing `messages` from being
identified as Chat Completions. Responses and Embeddings requests both use
`input`, so a request containing `input` but not `messages` is identified as
Embeddings unless its URI ends in `/v1/responses`. If no earlier rule matches,
the Plugin treats a non-empty JSON object or array as passthrough. Empty or
invalid request bodies are rejected.
+
+Requests that include `Content-Type` must use `application/json`. If the
header is omitted, the Plugin treats the body as JSON. The request body cannot
exceed `max_req_body_size`, which defaults to 67,108,864 bytes; a larger body
is rejected with HTTP `413`.
+
+Passthrough does not provide a named AI protocol to downstream AI-aware
Plugins. Usage extraction, prompt decoration, content-moderation text
extraction, and protocol conversion therefore do not run for a passthrough
body. The `ai-prompt-template` Plugin runs earlier in the rewrite phase and can
replace the body before protocol detection, so the rendered body can match a
named protocol or remain passthrough.
+
+For a named protocol, the Plugin uses it directly when the selected provider
supports it. Otherwise, it uses a registered converter to a protocol supported
by the provider. The request is rejected when neither native support nor a
compatible converter is available.
+
+### Chat Completions Request Format
+
| Name | Type | Required | Description
|
| ------------------ | ------ | -------- |
--------------------------------------------------- |
| `messages` | Array | True | An array of message objects.
|
@@ -67,7 +89,7 @@ When `provider` is set to `bedrock`, the Plugin expects
requests in the [Bedrock
| Name | Type | Required | Default | Valid values
| Description |
|--------------------|--------|----------|---------|------------------------------------------|-------------|
-| provider | string | True | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock,
openai-compatible] | LLM service provider. When set to `openai`, the Plugin
will proxy the request to `https://api.openai.com/chat/completions`. When set
to `deepseek`, the Plugin will proxy the request to
`https://api.deepseek.com/chat/completions`. When set to `aimlapi`, the Plugin
uses the OpenAI-compatible driver and proxies the request to [...]
+| provider | string | True | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock,
openai-compatible] | LLM service provider. When set to `openai`, the Plugin
sends detected Chat Completions, Responses, and Embeddings requests to the
corresponding OpenAI endpoint. When set to `deepseek`, the Plugin will proxy
the request to `https://api.deepseek.com/chat/completions`. When set to
`aimlapi`, the Plugin uses the OpenAI-compatib [...]
| provider_conf | object | False | |
| Configuration for the specific provider. Required when
`provider` is set to `vertex-ai` and `override` is not configured. Required
when `provider` is set to `bedrock`. |
| provider_conf.project_id | string | True | |
| Google Cloud Project ID. |
| provider_conf.region | string | True (depending on provider) | |
minLength = 1 (for Bedrock) | When `provider` is `vertex-ai`, this
is the Google Cloud Region. When `provider` is `bedrock`, this is the AWS
region used to construct the Bedrock endpoint and to sign the request with
SigV4 (required, must be non-empty). |
@@ -935,9 +957,56 @@ curl "http://127.0.0.1:9080/bedrock/converse"; -X POST \
}' --output -
```
+### Proxy OpenAI Responses API
+
+The following example configures the `ai-proxy` Plugin to proxy OpenAI
Responses API requests. Responses and Embeddings requests both contain `input`,
so the Route URI must end in `/v1/responses` for the Plugin to identify the
request as Responses.
+
+Obtain an [OpenAI API key](https://platform.openai.com/api-keys) and save it
to an environment variable:
+
+```shell
+export OPENAI_API_KEY="<your-api-key>"
+```
+
+Create a Route with the `/v1/responses` URI:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "ai-proxy-responses-route",
+ "uri": "/v1/responses",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-proxy": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4.1"
+ }
+ }
+ }
+ }'
+```
+
+Send a request using the Responses API format:
+
+```shell
+curl "http://127.0.0.1:9080/v1/responses"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "input": "Explain API gateways in one sentence."
+ }'
+```
+
+The Plugin sends the request to the OpenAI Responses endpoint and returns the
response in the Responses API format. To stream the response, add `"stream":
true` to the request body and use `curl --no-buffer`.
+
### Proxy to OpenAI Embedding Models
-The following example demonstrates how you can configure the `ai-proxy` Plugin
to proxy requests to embedding models. This example will use the OpenAI
embedding model endpoint.
+The following example demonstrates how you can configure the `ai-proxy` Plugin
to proxy requests to an OpenAI embedding model. The Plugin detects the `input`
field and selects the OpenAI Embeddings endpoint.
Obtain the OpenAI [API key](https://openai.com/blog/openai-api) and save it to
an environment variable:
@@ -976,9 +1045,6 @@ curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
"options":{
"model": "text-embedding-3-small",
"encoding_format": "float"
- },
- "override": {
- "endpoint": "https://api.openai.com/v1/embeddings";
}
}
}
@@ -1009,8 +1075,6 @@ services:
options:
model: text-embedding-3-small
encoding_format: float
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
```
Synchronize the configuration to the gateway:
@@ -1050,8 +1114,6 @@ spec:
options:
model: text-embedding-3-small
encoding_format: float
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
@@ -1105,8 +1167,6 @@ spec:
options:
model: text-embedding-3-small
encoding_format: float
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
```
</TabItem>
diff --git a/docs/en/latest/plugins/ai-rag.md b/docs/en/latest/plugins/ai-rag.md
index 113ee2bb26..f72c23c511 100644
--- a/docs/en/latest/plugins/ai-rag.md
+++ b/docs/en/latest/plugins/ai-rag.md
@@ -42,6 +42,10 @@ The `ai-rag` Plugin implements the retrieval step of a
Retrieval-Augmented Gener
The current implementation supports [Azure
OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
for embeddings and [Azure AI
Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) for
vector search. Use the [`ai-proxy`](./ai-proxy.md) Plugin in the same request
flow to proxy the augmented request to the LLM provider. The Plugin does not
create or populate a search index; prepare the index and its content before
sending requests through APISIX.
+The Plugin enriches Chat Completions, Responses API, Anthropic Messages, and
Bedrock Converse requests using each protocol's native prompt structure. For
Responses, it appends retrieved context to `input`. It leaves direct Embeddings
API and passthrough requests unchanged. The nested `ai_rag.embeddings` object
configures the embedding input used internally for retrieval and is distinct
from a top-level Embeddings request.
+
+See [request protocol detection](./ai-proxy.md#request-protocol-detection) for
the detection order and URI requirements.
+
## Plugin Attributes
| Name | Type | Required | Default | Valid values | Description |
diff --git a/docs/en/latest/plugins/ai-request-rewrite.md
b/docs/en/latest/plugins/ai-request-rewrite.md
index cdeddb0084..9579a8ea11 100644
--- a/docs/en/latest/plugins/ai-request-rewrite.md
+++ b/docs/en/latest/plugins/ai-request-rewrite.md
@@ -38,6 +38,8 @@ import TabItem from '@theme/TabItem';
The `ai-request-rewrite` Plugin processes client requests by forwarding them
to LLM services for transformation before relaying them to Upstream services.
This enables LLM-powered modifications such as data redaction, content
enrichment, or reformatting. The Plugin supports integration with OpenAI,
DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible
APIs.
+The LLM call used for rewriting is separate from the client's request format.
With the `openai` provider, the Plugin builds a non-streaming Chat Completions
request from the configured prompt and the original request body. It does not
classify or proxy the client's request as an AI protocol. The internal request
uses the Plugin's configured credentials and does not forward client headers.
+
## Plugin Attributes
| Name | Type | Required | Default | Valid values | Description |
@@ -48,7 +50,7 @@ The `ai-request-rewrite` Plugin processes client requests by
forwarding them to
| `auth` | object | True | | | Authentication configurations. |
| `auth.header` | object | False | | | Authentication headers. Key must match
pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be
configured. |
| `auth.query` | object | False | | | Authentication query parameters. Key
must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query`
should be configured. |
-| `options` | object | False | | | Model configurations. In addition to
`model`, you can configure additional parameters and they will be forwarded to
the upstream LLM service in the request body. For instance, if you are working
with OpenAI, you can configure additional parameters such as `temperature`,
`top_p`, and `stream`. See your LLM provider's API documentation for more
available options. |
+| `options` | object | False | | | Model configurations. In addition to
`model`, you can configure non-streaming generation parameters, such as
`temperature` and `top_p`, which are forwarded to the upstream LLM service. The
Plugin must receive a complete JSON response before it can replace the original
request body. |
| `options.model` | string | False | | | Name of the LLM model, such as
`gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more
available models. |
| `override` | object | False | | | Override setting. |
| `override.endpoint` | string | False | | | LLM provider endpoint. Required
when `provider` is `openai-compatible`. |
diff --git a/docs/zh/latest/plugins/ai-aliyun-content-moderation.md
b/docs/zh/latest/plugins/ai-aliyun-content-moderation.md
index 64c7297ec7..973f2ba025 100644
--- a/docs/zh/latest/plugins/ai-aliyun-content-moderation.md
+++ b/docs/zh/latest/plugins/ai-aliyun-content-moderation.md
@@ -45,6 +45,10 @@ import TabItem from '@theme/TabItem';
`ai-aliyun-content-moderation` 插件应与 [`ai-proxy`](./ai-proxy.md) 或
[`ai-proxy-multi`](./ai-proxy-multi.md) 插件配合使用以代理 LLM 请求。
+该插件使用每种协议的原生内容结构审核 Chat Completions、Responses API、Embeddings、Anthropic
Messages 和 Bedrock Converse 请求。对于 Responses，它会审核 `input`，并以 Responses API
格式返回被拦截的非流式或流式请求。对于 Embeddings，它会审核 `input` 中的字符串或字符串数组，并以 OpenAI
风格的错误响应返回被拒绝的请求。其他非空 JSON 请求体使用透传模式，插件不会从中提取特定格式的文本。
+
+检测顺序和 URI 要求参见[请求协议检测](./ai-proxy.md#请求协议检测)。
+
## 属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
@@ -65,7 +69,7 @@ import TabItem from '@theme/TabItem';
| response_check_service | string | 否 | `"llm_response_moderation"` | |
用于响应审核的阿里云服务。 |
| response_check_length_limit | number | 否 | `5000` | >= 1 |
响应内容长度上限。如果超过该限制，内容将分块发送到阿里云。例如，如果响应内容为 250 个字符，且 `response_check_length_limit`
设置为 `100`，则内容将分 3 次请求发送到阿里云。 |
| risk_level_bar | string | 否 | `"high"` | `none`、`low`、`medium`、`high`、`max`
| 如果评估的风险等级低于 `risk_level_bar`，请求或响应将分别被放行到上游 LLM 或客户端。 |
-| deny_code | integer | 否 | `200` | [200, 599] | 拒绝时的 HTTP 状态码。默认为 `200`，使兼容
provider 的拒绝响应在客户端 SDK 中被解析为正常补全；设置为 4xx 可将拒绝暴露为 HTTP 错误。 |
+| deny_code | integer | 否 | `200` | [200, 599] | 拒绝时的 HTTP 状态码。默认值 `200`
会以检测到的协议响应结构返回拒绝消息。设置为 4xx 可将拒绝呈现为 HTTP 错误。 |
| deny_message | string | 否 | | | 拒绝时的消息。 |
| timeout | integer | 否 | `10000` | >= 1 | 超时时间（毫秒）。 |
| keepalive | boolean | 否 | `true` | | 如果为 `true`，启用到阿里云的 HTTP 连接保活。 |
diff --git a/docs/zh/latest/plugins/ai-aws-content-moderation.md
b/docs/zh/latest/plugins/ai-aws-content-moderation.md
index d85fc82012..a0f06e3ad4 100644
--- a/docs/zh/latest/plugins/ai-aws-content-moderation.md
+++ b/docs/zh/latest/plugins/ai-aws-content-moderation.md
@@ -40,7 +40,7 @@ import TabItem from '@theme/TabItem';
`ai-aws-content-moderation` 插件集成了 [AWS
Comprehend](https://aws.amazon.com/comprehend/)，用于在代理请求到 LLM
时检查请求内容中的有害内容，例如亵渎、仇恨言论、侮辱、骚扰、暴力等，如果评估结果超过配置的阈值则拒绝请求。
-该插件是协议感知的：它会从 LLM 请求中提取提示内容（例如
`messages[].content`），仅审核解码后的文本，而不是原始请求体。`request_check_roles`
用于选择审核哪些消息角色，`request_check_mode` 可将 `user`/`tool` 的审核范围收窄到最新一轮，避免每次请求都重复审核历史对话。
+该插件支持 Chat Completions、Responses API、Embeddings、Anthropic Messages 和 Bedrock
Converse
请求。它会使用检测到的协议的原生内容结构提取文本，并仅审核解码后的文本，而不是原始请求体。被拒绝的请求使用检测到的协议对应的响应格式。`request_check_roles`
用于选择审核哪些消息角色，`request_check_mode` 可将 `user`/`tool`/`assistant`
的审核范围收窄到最新一轮，避免每次请求都重复审核历史对话。
`ai-aws-content-moderation` 插件应与 [`ai-proxy`](./ai-proxy.md) 或
[`ai-proxy-multi`](./ai-proxy-multi.md) 插件一起使用，以代理 LLM 请求。
diff --git a/docs/zh/latest/plugins/ai-cache.md
b/docs/zh/latest/plugins/ai-cache.md
index 794313ff4a..2edb85cd48 100644
--- a/docs/zh/latest/plugins/ai-cache.md
+++ b/docs/zh/latest/plugins/ai-cache.md
@@ -45,6 +45,8 @@ import TabItem from '@theme/TabItem';
- **精确缓存（L1）：** 对有效提示词计算 SHA-256 指纹并用作 Redis 键。完全相同的提示词始终命中同一条缓存条目。
- **语义缓存（L2）：** 当 L1 未命中时，将提示词向量化，并通过最近邻搜索检索相似度在阈值以上的历史响应。L2 默认关闭；在 `layers`
中加入 `"semantic"` 即可启用。
+精确缓存支持 Chat Completions、Responses API、Embeddings、Anthropic Messages、Bedrock
Converse 和透传请求，并将不同协议存储在独立的缓存条目中。对于透传请求，缓存键还包含请求方法、URI 和查询字符串。语义缓存仅适用于 Chat
Completions 请求；其他协议可以使用精确缓存层，但会绕过语义缓存层。
+
`ai-cache` 插件必须与 [`ai-proxy`](./ai-proxy.md) 或
[`ai-proxy-multi`](./ai-proxy-multi.md) 插件一起使用。
### 流式响应
diff --git a/docs/zh/latest/plugins/ai-lakera-guard.md
b/docs/zh/latest/plugins/ai-lakera-guard.md
index 46750da4d6..3d6adbbbd3 100644
--- a/docs/zh/latest/plugins/ai-lakera-guard.md
+++ b/docs/zh/latest/plugins/ai-lakera-guard.md
@@ -43,7 +43,9 @@ import TabItem from '@theme/TabItem';
运行哪些检测器以及使用何种阈值，完全由通过 `project_id` 选择的 **Lakera 项目策略**控制。网关侧没有检测器列表；Lakera
每次调用返回单一的判定结果。
-`ai-lakera-guard` 插件应与 [`ai-proxy`](./ai-proxy.md) 或
[`ai-proxy-multi`](./ai-proxy-multi.md) 插件配合使用以代理 LLM 请求。它依赖 `ai-proxy`
填充的上下文，以协议感知的方式提取对话内容。
+`ai-lakera-guard` 插件应与 [`ai-proxy`](./ai-proxy.md) 或
[`ai-proxy-multi`](./ai-proxy-multi.md) 插件配合使用以代理 LLM 请求。它依赖 `ai-proxy`
填充的上下文，以协议感知的方式提取请求内容。
+
+请求扫描支持 Chat Completions、Responses API、Embeddings、Anthropic Messages 和 Bedrock
Converse 请求。对于 Responses，插件会将 `instructions` 和 `input` 中的文本转换为对话消息。对于
Embeddings，插件会扫描 `input` 中的文本。响应扫描适用于返回生成文本的协议；Embeddings
响应包含向量而不是文本。插件不会扫描透传请求或响应。
未经过 `ai-proxy`/`ai-proxy-multi` 的请求（例如插件绑定在 Consumer 或 Service 级别时的普通 HTTP
流量）无法被检查。默认情况下，此类请求会被直接放行而不做检查；该行为可通过 `fail_mode` 配置。
@@ -63,7 +65,7 @@ import TabItem from '@theme/TabItem';
| timeout | integer | 否 | `5000` | >= 1 | Lakera 请求超时时间（毫秒）。 |
| ssl_verify | boolean | 否 | `true` | | 如果为 `true`，则验证 Lakera 端点的 TLS 证书。 |
| reveal_failure_categories | boolean | 否 | `false` | | 如果为 `true`，将匹配到的
Lakera `detector_type`（及其置信度结果）追加到返回给客户端的拒绝消息中。无论该设置如何，插件始终会向 Lakera
请求完整的每个检测器的 `breakdown` 并写入网关日志；此标志仅控制面向客户端的暴露。 |
-| deny_code | integer | 否 | `200` | 200 - 599 | 请求被拦截时返回的 HTTP 状态码。默认为
`200`，使响应体——一个携带 `request_failure_message` 的、与提供商兼容的聊天补全（或 SSE）——在客户端 SDK
中被解析为正常的拒绝消息（与 Lakera Guard 自身返回 `200` 并附带判定结果的方式一致）。如果你希望拦截以 HTTP 错误的形式呈现，可设置为
4xx（例如 `403`）。 |
+| deny_code | integer | 否 | `200` | 200 - 599 | 流量被拦截时返回的 HTTP 状态码。默认值 `200`
会以检测到的协议响应结构返回拒绝消息，并携带 `request_failure_message` 或
`response_failure_message`。设置为 4xx（例如 `403`）可将拒绝呈现为 HTTP 错误。 |
| request_failure_message | string | 否 | `Request blocked by Lakera Guard` | |
请求被拦截时返回的拒绝文本（作为与提供商兼容的响应中的 assistant 消息）。 |
| response_failure_message | string | 否 | `Response blocked by Lakera Guard` |
| LLM 响应被拦截时（`direction` 为 `output` 或 `both`）返回的拒绝文本（作为与提供商兼容的响应中的 assistant
消息）。 |
diff --git a/docs/zh/latest/plugins/ai-prompt-decorator.md
b/docs/zh/latest/plugins/ai-prompt-decorator.md
index bd32a5a2b8..92b37f6ada 100644
--- a/docs/zh/latest/plugins/ai-prompt-decorator.md
+++ b/docs/zh/latest/plugins/ai-prompt-decorator.md
@@ -38,6 +38,10 @@ import TabItem from '@theme/TabItem';
`ai-prompt-decorator`
插件通过在用户输入提示前后添加预设提示来修改用户输入，以在内容生成中设定上下文。这种做法有助于模型在交互过程中按照预期的指导方针运行。
+该插件使用每种协议的原生提示结构修饰 Chat Completions、Responses API、Anthropic Messages 和 Bedrock
Converse 请求。对于 Responses，`prepend` 中的内容会添加到 `instructions`，`append` 中的内容会添加到
`input`。Embeddings 和透传请求没有可前置或追加的提示角色，因此插件不会修改这两类请求。
+
+检测顺序和 URI 要求参见[请求协议检测](./ai-proxy.md#请求协议检测)。
+
## 插件属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
diff --git a/docs/zh/latest/plugins/ai-prompt-guard.md
b/docs/zh/latest/plugins/ai-prompt-guard.md
index 672f9e6db5..34224e58a1 100644
--- a/docs/zh/latest/plugins/ai-prompt-guard.md
+++ b/docs/zh/latest/plugins/ai-prompt-guard.md
@@ -40,6 +40,10 @@ import TabItem from '@theme/TabItem';
当同时配置了 `allow_patterns` 和 `deny_patterns` 时，插件首先确保至少匹配一个
`allow_patterns`。如果没有匹配，请求将被拒绝。如果匹配了允许的模式，它会继续检查是否存在任何拒绝模式的匹配。
+该插件使用每种协议的原生内容结构检查 Chat Completions、Responses API、Embeddings、Anthropic
Messages 和 Bedrock Converse 请求。对于 Responses，它会检查 `input` 中的用户内容；当
`match_all_roles` 为 `true` 时，还会检查 `instructions` 中的系统内容。
+
+检测顺序和 URI 要求参见[请求协议检测](./ai-proxy.md#请求协议检测)。其他非空 JSON 请求体使用透传模式，不会作为受支持的 AI
请求接受检查，而是根据 `fail_mode` 处理。
+
对于 Embeddings 请求，插件会检查字符串形式的 `input`，或 `input` 数组中的所有字符串。批量输入会用空格连接后一起检查，不受
`match_all_conversation_history` 影响。允许模式作用于连接后的文本；批量输入中任何位置匹配拒绝模式都会导致请求被拒绝。
当没有选中需要检查的内容时，请求会继续转发到上游。这包括被角色过滤排除的消息，以及仅包含 token ID 的 Embeddings 输入；插件不会将
token ID 解码为文本。`fail_mode: "error"` 适用于未识别的协议，不会仅因已识别请求没有选中文本而拒绝请求。
diff --git a/docs/zh/latest/plugins/ai-prompt-template.md
b/docs/zh/latest/plugins/ai-prompt-template.md
index 60617e8bf7..e1db666190 100644
--- a/docs/zh/latest/plugins/ai-prompt-template.md
+++ b/docs/zh/latest/plugins/ai-prompt-template.md
@@ -38,13 +38,15 @@ import TabItem from '@theme/TabItem';
`ai-prompt-template` 插件支持预先配置提示词模板，这些模板仅接受用户在指定的模板变量中输入，采用"填空"的方式。它简化了对
OpenAI、Anthropic 等大语言模型提供商及其模型的访问，让你可以定义可复用的提示词结构。
+该插件会使用选定并渲染后的 JSON 模板替换传入的请求体。模板可以使用下游插件支持的任意请求格式，包括 Chat
Completions、Responses API 和 Embeddings。如果需要在不替换请求体的情况下向现有请求添加内容，请改用
[`ai-prompt-decorator`](./ai-prompt-decorator.md)。
+
## 插件属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
| --- | --- | --- | --- | --- | --- |
| `templates` | array | 是 | | | 模板对象数组。 |
| `templates.name` | string | 是 | | | 模板的名称。在请求路由时，请求中应包含与所配置模板相对应的模板名称。 |
-| `templates.template` | object | 是 | | | 模板规范。 |
+| `templates.template` | object | 是 | | | JSON 请求体模板。它可以包含下游插件接受的任意字段。 |
| `templates.template.model` | string | 否 | | | LLM 模型的名称，例如 `gpt-4` 或
`gpt-3.5`。更多可用模型请参阅 LLM 提供商的 API 文档。 |
| `templates.template.messages` | array[object] | 否 | | | 模板消息规范。 |
| `templates.template.messages.role` | string | 是 | | [`system`, `user`,
`assistant`] | 消息的角色。 |
@@ -600,3 +602,112 @@ curl "http://127.0.0.1:9080/openai-chat"; -X POST \
...
}
```
+
+### 配置 Responses API Web 搜索模板
+
+以下示例配置一个仅搜索 Apache APISIX 官方网站的 Responses API 模板。路由 URI 可以使用自定义前缀，但必须以
`/v1/responses` 结尾，以便 `ai-proxy` 将渲染后的请求体与 Embeddings 请求区分开。
+
+创建一个配置了 `ai-proxy` 和 `ai-prompt-template` 插件的路由：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/ai-prompt-template-responses";
-X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/template/v1/responses",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-proxy": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ }
+ },
+ "ai-prompt-template": {
+ "templates": [
+ {
+ "name": "Search APISIX documentation",
+ "template": {
+ "model": "gpt-4.1",
+ "tools": [
+ {
+ "type": "web_search",
+ "filters": {
+ "allowed_domains": ["apisix.apache.org"]
+ }
+ }
+ ],
+ "tool_choice": "required",
+ "input": "Search the official Apache APISIX website for
{{topic}} and summarize the result in one sentence."
+ }
+ }
+ ]
+ }
+ }
+ }'
+```
+
+向路由发送模板名称和 `topic` 的值：
+
+```shell
+curl "http://127.0.0.1:9080/template/v1/responses"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "Search APISIX documentation",
+ "topic": "what APISIX is"
+ }'
+```
+
+在 `ai-proxy` 处理请求之前，`ai-prompt-template` 会使用渲染后的模板替换传入的请求体：
+
+```json
+{
+ "model": "gpt-4.1",
+ "tools": [
+ {
+ "type": "web_search",
+ "filters": {
+ "allowed_domains": ["apisix.apache.org"]
+ }
+ }
+ ],
+ "tool_choice": "required",
+ "input": "Search the official Apache APISIX website for what APISIX is and
summarize the result in one sentence."
+}
+```
+
+APISIX 根据渲染后请求中的 `input` 字段和路由 URI 后缀将其识别为 Responses API 请求。你应该会收到 HTTP `200`
响应，其中的 `output` 包含已完成的 Web 搜索调用和带有 URL 引用的消息。生成的文本、搜索详情和引用偏移量会随请求变化；响应结构类似如下：
+
+```json
+{
+ "status": "completed",
+ "output": [
+ {
+ "type": "web_search_call",
+ "status": "completed",
+ "action": {
+ "type": "search"
+ }
+ },
+ {
+ "type": "message",
+ "status": "completed",
+ "role": "assistant",
+ "content": [
+ {
+ "type": "output_text",
+ "text": "<带有引用的生成摘要>",
+ "annotations": [
+ {
+ "type": "url_citation",
+ "title": "Apache APISIX",
+ "url": "https://apisix.apache.org/";
+ }
+ ]
+ }
+ ]
+ }
+ ]
+}
+```
diff --git a/docs/zh/latest/plugins/ai-proxy-multi.md
b/docs/zh/latest/plugins/ai-proxy-multi.md
index db6e43e198..413da5f779 100644
--- a/docs/zh/latest/plugins/ai-proxy-multi.md
+++ b/docs/zh/latest/plugins/ai-proxy-multi.md
@@ -44,6 +44,12 @@ import TabItem from '@theme/TabItem';
## 请求格式
+### 请求协议检测
+
+`ai-proxy-multi` 与 `ai-proxy` 使用相同的请求验证、协议检测、透传和转换行为。完整规则和 URI
要求参见[请求协议检测](./ai-proxy.md#请求协议检测)。
+
+### Chat Completions 请求格式
+
| 名称 | 类型 | 必选项 | 描述 |
| ------------------ | ------ | -------- |
--------------------------------------------------- |
| `messages` | Array | 是 | 消息对象数组。 |
@@ -77,7 +83,7 @@ import TabItem from '@theme/TabItem';
| balancer.key | string | 否 |
| | 当 `type` 为 `chash` 时使用。当 `hash_on` 设置为
`header` 或 `cookie` 时，需要 `key`。当 `hash_on` 设置为 `consumer` 时，不需要
`key`，因为消费者名称将自动用作键。 |
| instances | array[object] | 是 |
| | LLM 实例配置。 |
| instances.name | string | 是 |
| | LLM 服务实例的名称。该名称在 `instances`
中必须唯一，因为负载均衡、健康检查以及 `ai-rate-limiting` 等引用该实例的插件都以它作为实例标识。 |
-| instances.provider | string | 是 |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM 服务提供商。设置为
`openai` 时，插件将代理请求到 `api.openai.com`。设置为 `deepseek` 时，插件将代理请求到
`api.deepseek.com`。设置为 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`api.aimlapi.com`。设置为 `anthropic` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`api.anthropic.com`。设置为 `openrouter` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `openro [...]
+| instances.provider | string | 是 |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM 服务提供商。设置为
`openai` 时，插件会将检测到的 Chat Completions、Responses 和 Embeddings 请求发送到
`api.openai.com` 上对应的端点。设置为 `deepseek` 时，插件将代理请求到 `api.deepseek.com`。设置为
`aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.aimlapi.com`。设置为 `anthropic`
时，插件会将检测到的 Chat Completions 请求发送到 `api.anthropic.co [...]
| instances.provider_conf | object | 否 |
| | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置
`override` 时必填。当 `provider` 设置为 `bedrock` 时必填。 |
| instances.provider_conf.project_id | string | 是 |
| | Google Cloud 项目 ID。 |
| instances.provider_conf.region | string | 视提供商而定 |
| minLength = 1（Bedrock 时） | 当 `provider` 为 `vertex-ai`
时，此项为 Google Cloud 区域。当 `provider` 为 `bedrock` 时，此项为用于构造 Bedrock 端点并使用 SigV4
对请求进行签名的 AWS 区域（必填，不能为空）。 |
@@ -146,6 +152,7 @@ import TabItem from '@theme/TabItem';
| keepalive | boolean | 否 | true
| | 如果为 true，在请求 LLM 服务时保持连接活跃。 |
| keepalive_timeout | integer | 否 | 60000
| 大于或等于 1000 | 请求 LLM 服务时的请求超时时间（毫秒）。 |
| keepalive_pool | integer | 否 | 30
| | 连接 LLM 服务时的保活池大小。 |
+| streaming_flush_interval_ms | integer | 否 | 10
| 大于或等于 0 | 流式响应的后台刷新间隔，单位为毫秒。正值会定期刷新缓冲输出，以限制上游突发发送 token
时的客户端延迟。设置为 `0` 时同步刷新每个数据块。 |
| ssl_verify | boolean | 否 | true
| | 如果为 true，验证 LLM 服务的证书。 |
## 请求头转发
@@ -1872,6 +1879,70 @@
https://bedrock-runtime.us-east-1.amazonaws.com/model/arn%3Aaws%3Abedrock%3Aus-e
要启用流式响应，请使用相同的 Converse 请求体，并在其中加上 `"stream": true`。插件会将请求路由到 Bedrock 的
`/model/<model>/converse-stream` 接口，并将 AWS EventStream 帧原样转发给客户端。响应的
`Content-Type` 为 `application/vnd.amazon.eventstream`，客户端需自行解析二进制帧（多数 AWS SDK
已自动处理）。
+### 对 Responses API 请求进行负载均衡
+
+以下示例配置 `ai-proxy-multi`，在两个 OpenAI 模型之间分配 Responses API 请求。Responses 和
Embeddings 请求都包含 `input`，因此路由 URI 必须以 `/v1/responses` 结尾，插件才能将请求识别为 Responses。
+
+获取 [OpenAI API 密钥](https://platform.openai.com/api-keys)并保存到环境变量：
+
+```shell
+export OPENAI_API_KEY="<your-api-key>"
+```
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "ai-proxy-multi-responses-route",
+ "uri": "/v1/responses",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-proxy-multi": {
+ "instances": [
+ {
+ "name": "openai-responses-primary",
+ "provider": "openai",
+ "weight": 1,
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4.1"
+ }
+ },
+ {
+ "name": "openai-responses-secondary",
+ "provider": "openai",
+ "weight": 1,
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4.1-mini"
+ }
+ }
+ ]
+ }
+ }
+ }'
+```
+
+使用 Responses API 格式发送请求：
+
+```shell
+curl "http://127.0.0.1:9080/v1/responses"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "input": "用一句话解释 API 网关。"
+ }'
+```
+
+请求会被转发到其中一个已配置的 OpenAI 实例，并以 Responses API 格式返回响应。
+
### 代理到嵌入模型
以下示例演示了如何配置 `ai-proxy-multi` 插件以代理请求并在嵌入模型之间进行负载均衡。
@@ -1910,9 +1981,6 @@ curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
},
"options": {
"model": "text-embedding-3-small"
- },
- "override": {
- "endpoint": "https://api.openai.com/v1/embeddings";
}
},
{
@@ -1961,8 +2029,6 @@ services:
Authorization: "Bearer ${OPENAI_API_KEY}"
options:
model: text-embedding-3-small
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
- name: az-openai-instance
provider: openai-compatible
weight: 0
@@ -2014,8 +2080,6 @@ spec:
Authorization: "Bearer your-api-key"
options:
model: text-embedding-3-small
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
- name: az-openai-instance
provider: openai-compatible
weight: 0
@@ -2081,8 +2145,6 @@ spec:
Authorization: "Bearer your-api-key"
options:
model: text-embedding-3-small
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
- name: az-openai-instance
provider: openai-compatible
weight: 0
diff --git a/docs/zh/latest/plugins/ai-proxy.md
b/docs/zh/latest/plugins/ai-proxy.md
index 55a1f4811c..9d9261babc 100644
--- a/docs/zh/latest/plugins/ai-proxy.md
+++ b/docs/zh/latest/plugins/ai-proxy.md
@@ -44,6 +44,28 @@ import TabItem from '@theme/TabItem';
## 请求格式
+### 请求协议检测
+
+插件会先检测客户端请求协议，再选择兼容的上游端点。插件按以下顺序检查规则：
+
+| 客户端协议 | 检测条件 | 路由 URI |
+| --- | --- | --- |
+| Bedrock Converse | 请求体包含 `messages` 数组，并且请求 URI 以 `/converse` 结尾。 | URI
可以包含自定义前缀，但必须保留 `/converse` 后缀。 |
+| Anthropic Messages | 请求体为 JSON 对象或数组，并且请求 URI 以 `/v1/messages` 结尾。 | URI
可以包含自定义前缀，但必须保留 `/v1/messages` 后缀。 |
+| OpenAI Responses | 请求体包含 `input`，并且请求 URI 以 `/v1/responses` 结尾。 | URI
可以包含自定义前缀，但必须保留 `/v1/responses` 后缀。 |
+| OpenAI Chat Completions | 请求体包含 `messages` 数组。 | 路由匹配的任意 URI。 |
+| OpenAI Embeddings | 请求体包含 `input`，并且之前的规则均未匹配。 | 路由匹配的任意 URI。 |
+
+基于 URI 的规则在仅基于请求体的规则之前运行，以免包含 `messages` 的 Bedrock Converse 和 Anthropic
Messages 请求被识别为 Chat Completions。Responses 和 Embeddings 请求都使用 `input`，因此包含
`input` 但不包含 `messages` 的请求会被识别为 Embeddings，除非其 URI 以 `/v1/responses`
结尾。如果之前的规则均未匹配，插件会将非空 JSON 对象或数组视为透传请求。空请求体或无效请求体会被拒绝。
+
+包含 `Content-Type` 的请求必须使用 `application/json`。如果省略该请求头，插件会将请求体视为 JSON。请求体不能超过
`max_req_body_size`，其默认值为 67,108,864 字节；超过该限制的请求会收到 HTTP `413` 响应。
+
+透传模式不会向下游 AI 感知插件提供具名 AI
协议。因此，透传请求体不会执行用量提取、提示词装饰、内容审核文本提取或协议转换。`ai-prompt-template` 插件在 rewrite
阶段更早运行，并可在协议检测前替换请求体，因此渲染后的请求体可以匹配具名协议，也可以保持透传模式。
+
+对于具名协议，如果所选提供商原生支持该协议，插件会直接使用它；否则，插件会使用已注册的转换器转换为提供商支持的协议。如果既不支持原生协议，也没有兼容的转换器，请求将被拒绝。
+
+### Chat Completions 请求格式
+
| 名称 | 类型 | 必选项 | 描述 |
| ------------------ | ------ | -------- |
--------------------------------------------------- |
| `messages` | Array | 是 | 消息对象数组。 |
@@ -67,7 +89,7 @@ import TabItem from '@theme/TabItem';
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述
|
|--------------------|--------|----------|---------|------------------------------------------|-------------|
-| provider | string | 是 | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock,
openai-compatible] | LLM 服务提供商。当设置为 `openai` 时，插件将代理请求到
`https://api.openai.com/chat/completions`。当设置为 `deepseek` 时，插件将代理请求到
`https://api.deepseek.com/chat/completions`。当设置为 `aimlapi` 时，插件使用 OpenAI
兼容驱动程序，默认将请求代理到 `https://api.aimlapi.com/v1/chat/completions`。当设置为 `anthropic`
时，插件将代理请求到 `https://api.anthropic.com/v1/chat/completions`。当设置为 `openrou [...]
+| provider | string | 是 | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock,
openai-compatible] | LLM 服务提供商。当设置为 `openai` 时，插件会将检测到的 Chat
Completions、Responses 和 Embeddings 请求发送到对应的 OpenAI 端点。当设置为 `deepseek`
时，插件将代理请求到 `https://api.deepseek.com/chat/completions`。当设置为 `aimlapi` 时，插件使用
OpenAI 兼容驱动程序，默认将请求代理到 `https://api.aimlapi.com/v1/chat/completions`。当设置为
`anthropic` 时，插件会将检测到的 Chat Completions 请求发送到 `https://api.anthropi [...]
| provider_conf | object | 否 | |
| 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。当
`provider` 设置为 `bedrock` 时必填。 |
| provider_conf.project_id | string | 是 | |
| Google Cloud 项目 ID。 |
| provider_conf.region | string | 视提供商而定 | | minLength = 1（Bedrock 时）
| 当 `provider` 为 `vertex-ai` 时，此项为 Google Cloud 区域。当 `provider` 为
`bedrock` 时，此项为用于构造 Bedrock 端点并使用 SigV4 对请求进行签名的 AWS 区域（必填，不能为空）。 |
@@ -911,9 +933,56 @@ curl "http://127.0.0.1:9080/bedrock/converse"; -X POST \
}' --output -
```
+### 代理 OpenAI Responses API
+
+以下示例配置 `ai-proxy` 插件以代理 OpenAI Responses API 请求。Responses 和 Embeddings 请求都包含
`input`，因此路由 URI 必须以 `/v1/responses` 结尾，插件才能将请求识别为 Responses。
+
+获取 [OpenAI API 密钥](https://platform.openai.com/api-keys)并保存到环境变量：
+
+```shell
+export OPENAI_API_KEY="<your-api-key>"
+```
+
+创建 URI 为 `/v1/responses` 的路由：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "id": "ai-proxy-responses-route",
+ "uri": "/v1/responses",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-proxy": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4.1"
+ }
+ }
+ }
+ }'
+```
+
+使用 Responses API 格式发送请求：
+
+```shell
+curl "http://127.0.0.1:9080/v1/responses"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "input": "用一句话解释 API 网关。"
+ }'
+```
+
+插件会将请求发送到 OpenAI Responses 端点，并以 Responses API 格式返回响应。如需流式响应，请在请求体中添加
`"stream": true`，并使用 `curl --no-buffer`。
+
### 代理到 OpenAI 嵌入模型
-以下示例演示了如何配置 `ai-proxy` 插件以将请求代理到嵌入模型。此示例将使用 OpenAI 嵌入模型端点。
+以下示例演示如何配置 `ai-proxy` 插件以代理对 OpenAI 嵌入模型的请求。插件会检测 `input` 字段并选择 OpenAI
Embeddings 端点。
获取 OpenAI [API 密钥](https://openai.com/blog/openai-api)并保存到环境变量：
@@ -932,7 +1001,7 @@ values={[
<TabItem value="admin-api">
-创建路由并配置 `ai-proxy` 插件，将 `provider` 设置为 `openai`，指定嵌入模型名称，添加 `encoding_format`
参数以配置返回的嵌入向量为浮点数列表，并使用 `override` 将默认端点覆盖为 [嵌入 API
端点](https://platform.openai.com/docs/api-reference/embeddings)：
+创建路由并配置 `ai-proxy` 插件，将 `provider` 设置为 `openai`，指定嵌入模型名称，并添加 `encoding_format`
参数，将返回的嵌入向量配置为浮点数列表：
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
@@ -952,9 +1021,6 @@ curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
"options":{
"model": "text-embedding-3-small",
"encoding_format": "float"
- },
- "override": {
- "endpoint": "https://api.openai.com/v1/embeddings";
}
}
}
@@ -965,7 +1031,7 @@ curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
<TabItem value="adc">
-创建包含 `ai-proxy` 插件配置的路由，将 `provider` 设置为 `openai`，指定嵌入模型名称，添加
`encoding_format` 参数，并使用 `override` 将默认端点覆盖为 [嵌入 API
端点](https://platform.openai.com/docs/api-reference/embeddings)：
+创建包含 `ai-proxy` 插件配置的路由，将 `provider` 设置为 `openai`，指定嵌入模型名称，并添加
`encoding_format` 参数：
```yaml title="adc.yaml"
services:
@@ -985,8 +1051,6 @@ services:
options:
model: text-embedding-3-small
encoding_format: float
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
```
将配置同步到网关：
@@ -1026,8 +1090,6 @@ spec:
options:
model: text-embedding-3-small
encoding_format: float
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
@@ -1081,8 +1143,6 @@ spec:
options:
model: text-embedding-3-small
encoding_format: float
- override:
- endpoint: "https://api.openai.com/v1/embeddings";
```
</TabItem>
diff --git a/docs/zh/latest/plugins/ai-rag.md b/docs/zh/latest/plugins/ai-rag.md
index 7adffdbf2f..b98b796f07 100644
--- a/docs/zh/latest/plugins/ai-rag.md
+++ b/docs/zh/latest/plugins/ai-rag.md
@@ -42,6 +42,10 @@ import TabItem from '@theme/TabItem';
该插件支持使用 [Azure
OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
和 [Azure AI
Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search)
服务来生成嵌入和执行向量搜索。欢迎提交 PR 以引入对其他服务提供商的支持。
+该插件使用每种协议的原生提示结构增强 Chat Completions、Responses API、Anthropic Messages 和 Bedrock
Converse 请求。对于 Responses，它会将检索到的上下文追加到 `input`。它不会增强直接的 Embeddings API
请求或透传请求。嵌套的 `ai_rag.embeddings` 对象用于配置检索过程中内部使用的嵌入输入，与顶层 Embeddings 请求不同。
+
+检测顺序和 URI 要求参见[请求协议检测](./ai-proxy.md#请求协议检测)。
+
## 插件属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
diff --git a/docs/zh/latest/plugins/ai-request-rewrite.md
b/docs/zh/latest/plugins/ai-request-rewrite.md
index bd0c1c0585..f06697a3e9 100644
--- a/docs/zh/latest/plugins/ai-request-rewrite.md
+++ b/docs/zh/latest/plugins/ai-request-rewrite.md
@@ -38,6 +38,8 @@ import TabItem from '@theme/TabItem';
`ai-request-rewrite` 插件在将客户端请求转发到上游服务之前，先将请求发送到 LLM 服务进行转换处理。这使得 LLM
能够对请求进行数据脱敏、内容增强或格式转换等修改。该插件支持集成 OpenAI、DeepSeek、Gemini、Vertex
AI、Anthropic、OpenRouter 以及其他 OpenAI 兼容的 API。
+用于重写的 LLM 调用与客户端请求格式相互独立。使用 `openai` 提供商时，插件会根据配置的提示词和原始请求体构建非流式 Chat
Completions 请求。它不会将客户端请求识别为 AI 协议，也不会按 AI 协议代理该请求。内部请求使用插件配置的凭据，不会转发客户端请求头。
+
## 插件属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
@@ -47,7 +49,7 @@ import TabItem from '@theme/TabItem';
| `auth` | object | 是 | | | 身份验证配置。 |
| `auth.header` | object | 否 | | | 身份验证请求头。键必须匹配模式
`^[a-zA-Z0-9._-]+$`。`header` 和 `query` 至少需要配置其中一个。 |
| `auth.query` | object | 否 | | | 身份验证查询参数。键必须匹配模式
`^[a-zA-Z0-9._-]+$`。`header` 和 `query` 至少需要配置其中一个。 |
-| `options` | object | 否 | | | 模型配置。除了 `model` 之外，还可以配置其他参数，这些参数会在请求体中转发给上游
LLM 服务。例如，使用 OpenAI 时，可以配置 `temperature`、`top_p` 和 `stream` 等参数。更多可用选项请参阅 LLM
提供商的 API 文档。 |
+| `options` | object | 否 | | | 模型配置。除了 `model` 之外，还可以配置 `temperature` 和
`top_p` 等非流式生成参数，这些参数会转发给上游 LLM 服务。插件必须收到完整的 JSON 响应，才能替换原始请求体。 |
| `options.model` | string | 否 | | | LLM 模型名称，例如 `gpt-4` 或 `gpt-3.5`。更多可用模型请参阅
LLM 提供商的 API 文档。 |
| `override` | object | 否 | | | 覆盖设置。 |
| `override.endpoint` | string | 否 | | | LLM 提供商端点。当 `provider` 为
`openai-compatible` 时必填。 |
