hengyuss opened a new issue, #7383:
URL: https://github.com/apache/shenyu/issues/7383
# [Task][AI Proxy] Implement the extensible AI Provider layer
Parent issue: #7351
Depends on: #7381
## Background
Different LLM providers may claim compatibility with the OpenAI Chat
Completions protocol while still using different fields, model names,
endpoints, authentication methods, and supported capabilities.
These differences should be handled by a Provider layer instead of being
embedded in `AiProxyPlugin`, Protocol, or Transport.
## Goal
Implement the `ShenyuAiProvider` extension mechanism and the initial Provider
adapters used by AI Proxy.
## Responsibilities
The Provider layer is responsible for:
- declaring supported protocols and operations;
- resolving the upstream endpoint and path;
- applying authentication, headers, and query parameters;
- mapping normalized fields to provider-specific request fields;
- applying model name mappings and provider defaults;
- validating provider capabilities;
- handling provider-specific request and response differences;
- preserving fields that do not require transformation.
Provider differences include:
- `max_tokens` and `max_completion_tokens`;
- model name mappings;
- endpoint path differences;
- authentication methods and request headers;
- stream, tool calling, and JSON mode capabilities;
- token usage timing and format;
- provider-specific request parameters.
## Initial Implementations
Implement:
- `openai`;
- `openai-compatible`;
- `deepseek`.
If DeepSeek requires only configuration and field mapping, it may reuse the
generic OpenAI-compatible implementation as a provider profile.
## Boundaries
The Provider layer must not:
- parse the original client protocol directly;
- write the client response;
- establish or manage HTTP connections;
- implement retry or fallback policy.
## Acceptance Criteria
- [ ] Provider implementations use the shared internal and upstream models.
- [ ] Provider implementations do not depend on Spring AI clients or DTOs.
- [ ] Protocol and operation compatibility can be validated.
- [ ] Endpoint, authentication, headers, and model mappings are applied by
the
Provider.
- [ ] `max_tokens` and `max_completion_tokens` differences are handled by the
Provider.
- [ ] Provider-specific fields are preserved or mapped as required.
- [ ] Provider code does not execute HTTP requests directly.
- [ ] Adding another Provider does not require changing `AiProxyPlugin` or
Transport.
- [ ] Unit tests cover field mapping, authentication, endpoint resolution,
capability validation, and provider-specific parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
