Copilot commented on code in PR #13637:
URL: https://github.com/apache/apisix/pull/13637#discussion_r3503741471
##########
t/plugin/prometheus-ai-proxy.t:
##########
@@ -432,3 +432,74 @@
qr/apisix_llm_latency_bucket\{type="ttft",.*route_id="4",.*,node="openai-gpt4".*
GET /apisix/prometheus/metrics
--- response_body eval
qr/apisix_llm_latency_count\{type="total",.*route_id="4",.*,node="openai-gpt4".*request_type="ai_stream",request_llm_model="gpt-3",llm_model="gpt-4"\}
1/
+
+
+
+=== TEST 21: send a chat request whose model name exceeds the 128-byte label
cap
+--- request eval
+"POST /chat\n" .
+qq#{"messages":[{"role":"user","content":"What is 1+1?"}], "model": "@{[ 'a' x
200 ]}"}#
+--- more_headers
+X-AI-Fixture: prometheus/chat-basic.json
+--- error_code: 200
+
+
+
+=== TEST 22: request_llm_model label is truncated to 128 bytes (cardinality
DoS guard)
+--- request
+GET /apisix/prometheus/metrics
+--- response_body_like eval
+qr/apisix_llm_prompt_tokens\{.*request_llm_model="a{128}",llm_model="gpt-4"\}/
+--- response_body_unlike eval
+qr/request_llm_model="a{129}"/
+
+
+
+=== TEST 23: disable request_llm_model / llm_model labels via plugin_metadata
disabled_labels
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local _, body = t("/apisix/admin/plugin_metadata/prometheus",
+ ngx.HTTP_PUT,
+ [[{
+ "disabled_labels": {
+ "llm_prompt_tokens": ["request_llm_model", "llm_model"]
+ }
+ }]]
+ )
+ ngx.say(body)
+ }
+ }
+--- response_body
+passed
+
+
+
+=== TEST 24: send a chat request with a distinct model
+--- request
+POST /chat
+{"messages":[{"role":"user","content":"What is 1+1?"}], "model":
"distinct-model-aaa"}
+--- more_headers
+X-AI-Fixture: prometheus/chat-basic.json
+--- error_code: 200
+
+
+
+=== TEST 25: send another chat request with a different distinct model
+--- request
+POST /chat
+{"messages":[{"role":"user","content":"What is 1+1?"}], "model":
"distinct-model-bbb"}
+--- more_headers
+X-AI-Fixture: prometheus/chat-basic.json
+--- error_code: 200
+
+
+
+=== TEST 26: disabled_labels collapses distinct client models to one
empty-valued series
+--- request
+GET /apisix/prometheus/metrics
+--- response_body_like eval
+qr/apisix_llm_prompt_tokens\{.*request_llm_model="",llm_model=""\}/
+--- response_body_unlike eval
+qr/request_llm_model="distinct-model-/
Review Comment:
The `response_body_unlike` assertion is not scoped to the
`apisix_llm_prompt_tokens` metric. After setting `disabled_labels` only for
`llm_prompt_tokens`, other metrics (e.g., `apisix_http_status`,
`apisix_http_latency`, `apisix_llm_latency`) can still legitimately include
`request_llm_model="distinct-model-..."`, which can make this test fail even
when the label-collapse behavior is correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
