janiussyafiq opened a new issue, #13620:
URL: https://github.com/apache/apisix/issues/13620
### Current Behavior
When `ai-lakera-guard` streaming output scanning runs on the same route as a
response-filter plugin with **higher priority**, the other plugin can terminate
the response body-filter chain before `ai-lakera-guard` runs its end-of-stream
pass. Lakera's streaming output decision never executes, and the content it
buffered is stranded.
Mechanism:
- `apisix/plugin.lua` `lua_response_filter()` iterates plugins in
priority-descending order and, the moment any plugin's `lua_body_filter`
returns a code, prints that body and calls `ngx.exit()` — later plugins in the
chain do not run for that pass.
- `ai-aliyun-content-moderation` (priority **1029**) runs before
`ai-lakera-guard` (priority **1028**). In its default
`stream_check_mode=final_packet`, aliyun returns `ngx.OK` on the final packet —
even for a clean response — which triggers that `ngx.exit()`.
- As a result, `ai-lakera-guard` never receives the end-of-stream pass that
PR #13606 dispatches specifically so buffering plugins can flush/replace their
content.
Effect:
- `ai-lakera-guard` output scan is skipped (silent moderation gap).
- In `action=block` it is worse: lakera withholds every chunk (emitting only
SSE keep-alive comments) and buffers the real content to release at
end-of-stream. Since that final pass never happens, the buffered content is
never released — the client receives only keep-alives plus aliyun's `[DONE]`,
i.e. an effectively empty response.
### Expected Behavior
A buffering output-filter plugin should still get to make and apply its
end-of-stream decision when chained with another response-path plugin.
`ai-lakera-guard`'s streaming output scan should run and either release or
block its buffered content regardless of other response-filter plugins on the
route.
Note on the fix: a priority swap alone is only a partial fix —
`ai-lakera-guard` returns a nil code on the clean path (chain continues) but
`ngx.OK` on block, while `ai-aliyun-content-moderation` returns `ngx.OK`
unconditionally on the final packet. A robust fix should let buffering output
filters complete instead of terminating the chain on the first returned code
(and/or order lakera's output filter ahead of aliyun's `final_packet` filter).
A combined-plugin streaming test should be added.
### Error Logs
No errors are logged — both plugins report success. The failure is silent:
lakera's scan never runs and, in block mode, the buffered content is dropped.
### Steps to Reproduce
1. Run APISIX with PR #13606 applied.
2. Create a route with `ai-proxy` to an OpenAI-compatible streaming
upstream, plus both plugins (default priorities, no `_meta.priority` override):
- `ai-lakera-guard`: `direction=output` (or `both`), `action=block`
- `ai-aliyun-content-moderation`: `check_response=true`,
`stream_check_mode=final_packet` (default)
3. Send a streaming chat completion request (`"stream": true`).
4. Observe: lakera's output scan does not run; the client receives only SSE
keep-alive comments and `[DONE]`, not the assistant content. Configure Lakera
to flag the response and confirm it is neither blocked nor delivered as
expected.
### Environment
- APISIX version: `master` with PR #13606 (`ai-lakera-guard` response/output
scanning).
- Plugins involved: `ai-lakera-guard` (priority 1028) +
`ai-aliyun-content-moderation` (priority 1029).
- Code-path defect independent of OS / OpenResty / etcd versions;
reproducible on any environment running that branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
