lymerin commented on issue #7381: URL: https://github.com/apache/shenyu/issues/7381#issuecomment-5981894535
Hi @sunnysabor @hengyuss, I'm planning the Provider work for #7383. Using [draft #7397](https://github.com/apache/shenyu/pull/7397) as a reference, I'd like to check three points for the revised contracts. I won't change the shared contracts; any Provider-side work will stay internal and experimental until #7381 is updated. 1. `createRequest(ShenyuAiRequest)` takes no configuration argument. How should a Provider receive the resolved `baseUrl`, `apiKey`, and model settings for each attempt? Fallback can use a different endpoint and key. Would a separate per-call context—a resolved config snapshot passed alongside the request—fit the direction you have in mind for #7381, or should this live in the request model? 2. `decodeStream(AiUpstreamResponse)` currently receives raw `Flux<ByteBuffer>`, while #7384 assigns SSE framing to Protocol. Should the Provider receive already-decoded frames, such as event type and data, from a shared protocol decoder? I couldn't find that handoff in the current `ShenyuAiProtocol` interface. 3. `ShenyuAiRequest` carries both a normalized `maxTokens` and the original payload. Should the shared model preserve where the token limit came from—configuration or client input, and which key—or is the Provider expected to recover that information from the per-call context and raw payload? The existing adapter overwrites both `max_tokens` and `max_completion_tokens` when configured `maxTokens` is present; otherwise it keeps the client's fields without a local conflict check. I'd like to preserve that behavior in the first OpenAI implementation. For now, I'm assuming the `shenyu-plugin-ai-api` module direction from @sunnysabor's earlier note. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
