xuande opened a new issue, #12717:
URL: https://github.com/apache/apisix/issues/12717
### Description
There are two model services `DeepSeek-R1-Distill-Qwen-7B` and
`DeepSeek-R1-Distill-Qwen-14B`, with the request methods as follows：
```
curl -X POST "192.168.10.14:5111/v1/chat/completions" \
--header 'Authorization: Bearer token-123' \
-H "Content-Type: application/json" \
-d '{
"model": "DeepSeek-R1-Distill-Qwen-7B",
"messages": [
{"role": "user", "content": "what 1+1 = ?"}
],
"temperature": 0.7,
"max_tokens": 1
}'
```
```
curl -X POST "192.168.10.14:5112/v1/chat/completions" \
--header 'Authorization: Bearer token-123' \
-H "Content-Type: application/json" \
-d '{
"model": "DeepSeek-R1-Distill-Qwen-14B",
"messages": [
{"role": "user", "content": "what 1+1 = ?"}
],
"temperature": 0.7,
"max_tokens": 1
}'
```
It is required to unify these two model addresses into
`http://192.168.10.14:5000/v1/chat/completions`, and forward requests to
different model services based on the "model" value in the request body. For
example, when I send a request to `curl -X POST
"192.168.10.14:5000/v1/chat/completions" -d '{ "model":
"DeepSeek-R1-Distill-Qwen-14B" ...`, it is forwarded to
`192.168.10.14:5112/v1/chat/completions`.
### Environment
- APISIX version (run `apisix version`):
- Operating system (run `uname -a`):
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
- etcd version, if relevant (run `curl
http://127.0.0.1:9090/v1/server_info`):
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
