hengyuss opened a new issue, #7381:
URL: https://github.com/apache/shenyu/issues/7381
# [Task][AI Proxy] Define internal models and extension contracts
Parent issue: #7351
## Background
The AI Proxy currently uses OpenAI and Spring AI types directly in its main
execution path. Before Protocol, Provider, and Transport can be implemented
as
independent layers, they need a shared set of ShenYu-owned models and stable
interface contracts.
Without these contracts, each layer may introduce its own request, response,
and streaming representations, making the layers difficult to integrate and
extend.
## Goal
Define the protocol-neutral internal models and the extension contracts
shared
by Protocol, Provider, and Transport.
The models must not depend on Spring AI or a specific LLM provider.
## Scope
### Internal models
Define ShenYu-owned models for:
- AI requests;
- AI responses;
- streaming events;
- token usage;
- finish reasons;
- normalized errors;
- upstream transport requests;
- upstream transport responses.
The initial model definitions may be based on the request, response, usage,
error, and streaming structures already used by the current AI Proxy. This
allows the first version of the new model to preserve existing behavior while
replacing OpenAI- and Spring AI-specific types with ShenYu-owned types. The
models can then evolve as additional protocols and providers are introduced.
The request and response models should contain both:
- normalized fields required by ShenYu, such as operation, model, streaming
mode, token limit, token usage, and finish reason;
- the original payload or extensions, so unknown protocol- or
provider-specific fields are not lost.
The internal model should not attempt to represent every field supported by
every provider.
### Extension contracts
Define the interfaces and their input/output boundaries:
```text
ShenyuAiProtocol:
client protocol data <-> ShenyuAiRequest / ShenyuAiResponse /
ShenyuAiStreamEvent
ShenyuAiProvider:
ShenyuAiRequest -> AiUpstreamRequest
AiUpstreamResponse -> ShenyuAiResponse / ShenyuAiStreamEvent
ShenyuAiTransport:
AiUpstreamRequest -> AiUpstreamResponse
```
Define how implementations are identified and discovered through ShenYu SPI.
## Out of Scope
- Implementing a concrete Protocol.
- Implementing a concrete Provider.
- Implementing HTTP transport.
- Migrating `AiProxyPlugin` to the new execution path.
## Acceptance Criteria
- [ ] Internal models are owned by ShenYu and do not depend on Spring AI.
- [ ] Internal models do not expose OpenAI- or provider-specific DTOs.
- [ ] Requests and responses can retain unknown fields through raw payloads
or
extensions.
- [ ] Normal and streaming responses have explicit internal representations.
- [ ] Token usage and normalized error models are defined.
- [ ] Upstream transport request and response models are defined.
- [ ] `ShenyuAiProtocol`, `ShenyuAiProvider`, and `ShenyuAiTransport`
contracts
have clear input and output types.
- [ ] Extension implementations can be discovered through ShenYu SPI.
- [ ] Interface-level tests or examples are provided.
- [ ] Existing AI Proxy behavior is not changed by this task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
