hanzhe-one opened a new pull request, #7937: URL: https://github.com/apache/shenyu/pull/7937
## Problem The AI token limiter only recognized the Chat Completions usage layout. The enhanced AI proxy supports OpenAI Responses clients and converts their traffic, but the converted bodies and terminal stream events report usage as `input_tokens`/`output_tokens`, so the observer recorded zero tokens for a successful Responses completion while the same completion was charged 12 tokens in Chat format (issue #7911). Three places read only `completion_tokens`: - the streaming observer (`usage.completion_tokens` at the event root), - the buffered fallback pattern, - and, for Responses, the usage is not even at the event root — it sits inside the `response` object of the terminal event. ## Fix - Read the usage object from the event root (Chat) or from the `response` object (Responses), in both the streaming observer and the buffered fallback. - Accept `completion_tokens` or `output_tokens` as the output-token count. The fallback pattern keeps the leading quote so `max_output_tokens` — a request echo — is never mistaken for an accounted count. - Accounting policy is unchanged: the last reported output-token count is charged, and the response bytes are forwarded untouched. ## Tests - `AiTokenLimiterResponsesUsageTest` (new) drives the observer with the bytes produced by the real `ConnectProtocolAdapter`, not hand-written fixtures, and asserts that Chat and converted Responses bodies/streams record the same 12 tokens and that the converted bytes are forwarded unchanged. - `AiTokenLimiterPluginTest` gains Responses stream (single buffer and split across buffers, buffered and flush paths), complete-body, `max_output_tokens`-is-not-a-count, and no-usage cases. Both new integration tests fail on the previous code with `expected: <12> but was: <0>` and pass with the fix. `mvn -pl shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-token-limiter -am test -Dtest='AiTokenLimiter*Test'`: 30 tests, 0 failures, checkstyle clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
