hengyuss opened a new issue, #7532:
URL: https://github.com/apache/shenyu/issues/7532
# [Task][AI Proxy] Complete orchestration in AiProxyEngine
Parent issue: #7351
Depends on: #7382, #7383, #7384
## Background
#7381 introduces a minimal `AiProxyEngine` that validates the initial layer
boundaries by decoding an OpenAI Chat request, constructing an OpenAI
upstream
request, and delegating it to Transport.
After Protocol response handling, Provider adaptation, and production
Transport behavior are completed, the Engine must coordinate the complete
request and response lifecycle and own cross-layer execution policies.
## Goal
Complete `AiProxyEngine` as the protocol-independent orchestration layer for
both streaming and non-streaming AI exchanges.
## Responsibilities
- resolve the configured or detected Protocol;
- resolve and validate the selected Provider;
- coordinate Protocol request decoding and Provider request adaptation;
- execute the upstream request through Transport;
- coordinate Provider-specific response adaptation and Protocol response
encoding;
- use one orchestration flow for streaming and non-streaming requests;
- coordinate retry and fallback without moving those policies into Transport;
- distinguish failures that occur before a streaming response starts from
failures after downstream data has been emitted;
- propagate cancellation through the complete reactive chain;
- expose enough response status, headers, body, and error information for the
plugin boundary to write the downstream response;
- keep cross-layer policy out of Protocol, Provider, and Transport
implementations.
## Boundaries
`AiProxyEngine` must not:
- depend on `ServerWebExchange` or write the downstream HTTP response;
- contain OpenAI-, DeepSeek-, or other Provider-specific field mappings;
- parse protocol-specific SSE events directly;
- establish HTTP connections itself;
- hide upstream failures after streaming data has already been emitted.
## Acceptance criteria
- [ ] Streaming and non-streaming requests use the same Engine entry point.
- [ ] The complete Protocol → Provider → Transport → Provider → Protocol flow
is coordinated by `AiProxyEngine`.
- [ ] Retry and fallback occur only for eligible failures and do not
duplicate
data after a stream has started.
- [ ] Fallback requests are rebuilt using the fallback Provider
configuration.
- [ ] Cancellation reaches the active upstream Transport subscription.
- [ ] Upstream status, headers, bodies, and normalized failures remain
available at the plugin boundary.
- [ ] Engine code does not depend on Spring WebFlux server response types or
Spring AI clients and DTOs.
- [ ] Tests cover normal responses, streaming responses, retry, fallback,
cancellation, unsupported Protocol/Provider combinations, and failures
before and after stream emission.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
