eye-gu commented on code in PR #6341:
URL: https://github.com/apache/shenyu/pull/6341#discussion_r3254508733
##########
shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/src/main/java/org/apache/shenyu/plugin/ai/proxy/enhanced/service/AiProxyExecutorService.java:
##########
@@ -39,78 +45,124 @@ public class AiProxyExecutorService {
private static final Logger LOG =
LoggerFactory.getLogger(AiProxyExecutorService.class);
/**
- * Execute the AI call with retry and fallback.
+ * Execute a streaming AI call directly via {@link OpenAiApi}, bypassing
Spring AI's
+ * {@code createRequest()} which loses fields like {@code
reasoning_content}.
*
- * @param mainClient the main chat client
- * @param fallbackClientOpt the optional fallback chat client
- * @param requestBody the request body
- * @return a Mono containing the ChatResponse
+ * @param mainApi the main OpenAiApi
+ * @param fallbackCtxOpt the optional fallback context (api + config)
+ * @param request the ChatCompletionRequest with all fields
preserved
+ * @param requestBody the original request body for rebuilding fallback
request
+ * @param stream whether this is a streaming request
+ * @return a Flux of ChatCompletionChunk
*/
- public Mono<ChatResponse> execute(final ChatClient mainClient, final
Optional<ChatClient> fallbackClientOpt, final String requestBody) {
- final Mono<ChatResponse> mainCall = doChatCall(mainClient,
requestBody);
+ public Flux<ChatCompletionChunk> executeDirectStream(final OpenAiApi
mainApi,
+ final Optional<FallbackContext> fallbackCtxOpt, final
ChatCompletionRequest request,
+ final String requestBody, final boolean stream) {
+ return mainApi.chatCompletionStream(request)
+ .doOnError(e -> UpstreamErrorLogger.logUpstreamError(LOG, e,
"direct stream"))
+ .retryWhen(Retry.max(1)
Review Comment:
@moremind This is the original behavior — stream requests are limited to a
single retry. I guess the rationale is that streaming responses are typically
long-running; allowing multiple retries would accumulate excessive latency,
making the overall response time unacceptable. The isRetryable filter added in
this PR is an improvement over the original logic, but the max(1) cap remains
intentionally unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
