RGZingYang opened a new issue, #13769:
URL: https://github.com/apache/apisix/issues/13769
### Current Behavior
开启日志记录：
nginx_config:
http:
access_log_format: "$remote_addr - $remote_user [$time_local] $http_host
\"$request_line\" $status $body_bytes_sent $request_time \"$http_referer\"
\"$http_user_agent\" $upstream_addr $upstream_status
$apisix_upstream_response_time
\"$upstream_scheme://$upstream_host$upstream_uri\" \"$apisix_request_id\"
\"$request_type\" \"$llm_time_to_first_token\" \"$llm_model\"
\"$request_llm_model\" \"$llm_prompt_tokens\" \"$llm_completion_tokens\""
使用ai-proxy-multi创建路由 开启fallback_strategy， 发生重试后$request_llm_model不是原始请求的model
创建路由的curl：
```
curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \
-H "X-API-KEY: ${admin_key}" \
-d '{
"id": "ai-proxy-multi-route",
"uri": "/v1/chat/completions",
"methods": ["POST"],
"vars": [
[
"post_arg.model",
"==",
"model_test"
]
],
"plugins": {
"ai-proxy-multi": {
"instances": [
{
"provider": "openai-compatible",
"options": {
"model": "upstream-model-A"
},
"override": {
"endpoint":
"http://172.0.0.1:8081/test-upstream/v1/chat/completions";
},
"priority": 1,
"name": "test1",
"auth": {},
"weight": 1
},
{
"provider": "openai-compatible",
"options": {
"model": "upstream-model-B"
},
"override": {
"endpoint":
"http://172.0.0.1:8081/test-upstream/v1/chat/completions";
},
"priority": 0,
"name": "test2",
"auth": {},
"weight": 1
}
],
"balancer": {
"hash_on": "consumer",
"algorithm": "chash"
},
"timeout": 500,
"fallback_strategy": [
"http_429",
"http_5xx"
],
"logging": {
"summaries": true,
"payloads": true
},
"keepalive": false
},
}
}'
```
步骤：
发送请求：{
"model": "model_test",
"messages": [
{
"role": "user",
"content": "返回1"
}
]
}
上游正常返回 $request_llm_model 输出的是请求中的model参数 model_test $llm_model输出
upstream-model-A 符合预期
<img width="1630" height="26" alt="Image"
src="https://github.com/user-attachments/assets/112db910-e300-438e-b40f-fbd759723b59";
/>
发生重试后 $request_llm_model 输出为 upstream-model-A $llm_model输出
upstream-model-B 不符合预期 $request_llm_model应还是输出 model_test ：<img width="1960"
height="32" alt="Image"
src="https://github.com/user-attachments/assets/2372602b-8478-41bc-b701-3d5c4d6d8bab";
/>
### Expected Behavior
$request_llm_model 应是保持原始的模型名称 重试不应该改变， 即使为了兼容不使用 "vars":
[["post_arg.model","==","model_test"]，请求参数不带model 也应该在赋值的时候判断下 是否为空
如果不为空的情况下不重新赋值
### Error Logs
重试会调用base.before_proxy的方法重试
if request_model then
ctx.var.request_llm_model = request_model
end
赋值的时候需要添加判断是否已经赋过值了<img width="1044" height="900" alt="Image"
src="https://github.com/user-attachments/assets/0997ab09-fe30-4350-b21f-0d2924b32011";
/>
### Steps to Reproduce
步骤如描述中
### Environment
- APISIX version (run `apisix version`): 3.17.0
- Operating system (run `uname -a`):
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
- etcd version, if relevant (run `curl
http://127.0.0.1:9090/v1/server_info`):
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
