janiussyafiq opened a new issue, #13353: URL: https://github.com/apache/apisix/issues/13353
### Context APISIX currently ships moderation plugins that target the AI request/response path: - `ai-aws-content-moderation` (priority 1050) - `ai-aliyun-content-moderation` (priority 1029) - `ai-lakera-guard` (priority 1028, proposed in [api7/rfcs#32](https://github.com/api7/rfcs/pull/32)) They all run in the `access` and `lua_body_filter` phases. ### Problem When an operator configures multiple moderation plugins on the same route (defense-in-depth), each plugin independently calls its own vendor API for every request and every streaming body chunk. There is no shared signal that another moderation plugin has already decided *"this content is OK"* or *"this content is flagged."* Concrete consequences: 1. **Vendor calls are additive** — 2× or N× cost per request and per buffer flush. 2. **Last-flagger-wins deny body shape.** Each `lua_body_filter` plugin reads original upstream content from `ctx.var.llm_response_text` / `ctx.llm_response_contents_in_chunk` regardless of earlier plugins' body rewrites, makes its own independent flag decision, and rewrites the body if it flags. When multiple plugins flag, the client sees whichever plugin's deny shape ran last (Aliyun-shaped one moment, Lakera-shaped the next). 3. **Operators get neither pass nor block coordination.** No "fast pass" (skip remaining vendors on clean) nor "first block wins" (skip remaining vendors on flag). ### Proposed design Introduce shared `ctx.var` signals consumed by all moderation plugins: - `ctx.var.ai_moderation_decided` (boolean) — set to `true` by the first plugin that produces a verdict, clean or flagged. - `ctx.var.ai_moderation_flagged` (boolean) — set to `true` by the first plugin that flags. Each moderation plugin gains a config knob `coordinate_with_siblings: bool`, **default `false`** to preserve current independent-scan behavior (no breaking change for existing setups). When set `true`: - If `ai_moderation_decided` is already `true`, the plugin skips its scan and inherits the prior decision. - If `ai_moderation_flagged` is `true`, the plugin lets the earlier plugin's deny shape stand (or returns its own — design decision worth a sub-discussion). - If clean, the plugin lets the request through without calling its vendor. ### Out of scope Cross-vendor verdict reconciliation (e.g., Aliyun says clean but Lakera says flag — should the operator be alerted to disagreement, treated as flagged, treated as clean?). That's a follow-up if anyone deploys this pattern at scale and reports concrete needs. ### Discovered While drafting [api7/rfcs#32](https://github.com/api7/rfcs/pull/32) §4.7 (`ai-lakera-guard` composition with sibling moderation plugins). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
