yuluo-yx opened a new pull request, #7575: URL: https://github.com/apache/shenyu/pull/7575
## Summary Integrate [Connect](https://github.com/agentic-ai-java/connect) as the AI Proxy's protocol-conversion library, pinned to the Java 17-compatible Maven Central release `io.github.agentic-ai-java:connect:0.1.0`. ShenYu stays the reactive API gateway. Connect is a JSON codec/provider-adaptation dependency, **not** a replacement HTTP client, gateway, billing service, or agent runtime. This change adds OpenAI Responses clients on top of existing Chat Completions upstreams, delegates explicit DeepSeek dialect adaptation to Connect, and preserves existing generic Chat routes. It does **not** expose all of Connect's protocol pairs: Claude, Gemini, and native Responses upstream transports are outside this PR. ## Project boundary and responsibilities | Concern | Connect owns | ShenYu owns | | --- | --- | --- | | Protocol semantics | Request/response JSON mapping, messages, tools/tool calls, reasoning, finish reasons and usage | HTTP request-envelope validation and client-protocol selection | | Provider dialect | Applying an explicitly selected provider profile once to the final request | Provider/model routing and gateway parameter overrides | | Conversion fidelity | Structured diagnostics and `SAFE` loss-policy enforcement | Error classification, HTTP failure handling and diagnostic-code logging without payloads/secrets | | Tool restoration | Request-context mappings used to restore tool names/types | Isolated context allocation and binding each response to its originating request | | Streaming | Stateful conversion of JSON chunks into protocol events | Reactive HTTP transport, incremental SSE byte parsing/framing, backpressure, cancellation and memory limits | | Gateway execution | No networking or gateway policy | Configuration sync, selectors/rules, keys/authentication, endpoint construction, HTTP status/headers, retry and fallback | | Billing/agent execution | Usage normalization only; no charging or tool execution | Gateway quota/billing policy remains a gateway concern; this PR adds neither billing nor an agent/tool executor | Protocol field mappings should evolve in Connect rather than be copied into ShenYu providers. A new upstream protocol still needs a ShenYu provider/transport contract for its endpoint, authentication, status handling and lifecycle. ## How they work together 1. ShenYu resolves the selector/global configuration, credentials, routed model and fallback. 2. `OpenAiChat` or the new `OpenAiResponses` validates the client envelope and retains the original JSON. 3. `OpenAi` applies gateway overrides. `ConnectProtocolAdapter` then converts to Chat with a request-scoped `ConversionContext` and the explicitly selected profile. The configured upstream model is authoritative; the profile runs once. 4. ShenYu constructs the existing `/v1/chat/completions` HTTP request and sends it through `WebClientAiProxyTransport`. 5. `AiUpstreamRequest` binds a response adapter to the same conversion context. Successful Chat responses are converted back to Responses with original custom-tool mappings. Generic Chat responses and non-success upstream HTTP responses remain passthrough. 6. For Responses streams, ShenYu's existing incremental SSE framer extracts upstream JSON, Connect converts it, and ShenYu emits `event: <type>\ndata: <json>\n\n`. Conversion stream state is created per subscription. Normal completion finalizes it; cancellation or transport failure does not synthesize success. 7. Retry/fallback remains in `AiProxyEngine`. Fallback changes the upstream but preserves the already selected client protocol. No retry/fallback occurs after a converted event has been emitted. ### Configuration and usage Use the existing AI Proxy starter. Both protocol beans are registered automatically. Opt in through the selector handle's existing `protocol` field; its default remains `openai-chat`: ```json { "provider": "OpenAI", "baseUrl": "https://api.openai.com";, "apiKey": "<upstream-api-key>", "model": "gpt-4o-mini", "protocol": "openai-responses", "stream": false } ``` Send a request to a gateway path matched by that selector: ```json { "model": "client-model-alias", "input": "Explain reactive backpressure briefly.", "max_output_tokens": 128, "stream": true } ``` The upstream receives Chat `messages`, the routed model and `max_completion_tokens`; the client receives Responses JSON/SSE and converted usage when supplied upstream. An explicit client `stream` value overrides the configured default. Gateway `temperature`/`maxTokens` overrides are applied before conversion. Only the explicit `DeepSeek` provider identifier enables `ProviderProfile.deepSeekChat()`. `ALiYun`, `OpenAPI` and `Moonshot` remain generic Chat-compatible providers. URLs/model names never implicitly enable a dialect. See [`shenyu-plugin-ai-proxy/README.md`](https://github.com/yuluo-yx/shenyu/blob/refactor/ai-proxy-connect/shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/README.md) for the complete boundary, lifecycle and migration documentation. ## Compatibility and limitations - Generic Chat request bytes remain unchanged when no gateway override, stream-usage addition or profile adaptation changes the JSON. Extension fields survive identity conversion. Existing Chat endpoint/authentication and response/SSE behavior are retained. - `SAFE` rejects error-level conversion losses before HTTP dispatch (400), including Responses conversation state that Chat cannot represent. Warning-level conversions are allowed and logged by code, severity and formats, not library diagnostic messages/payloads. This is not a stateful Responses service or hosted-tool implementation. - Explicit DeepSeek routes now use stricter dialect validation and must review incompatible thinking/tool configurations. - Complete Responses conversion buffers at most 5 MiB. Responses streaming conversion limits cumulative framed upstream data to 5 MiB because Connect retains output/tool state. Oversize conversion fails; a committed stream aborts. No new cumulative-response limit is imposed on Chat passthrough. - Converted representation headers remove stale length/digest/ETag values. Conversion failures are not retried. Usage normalization must not be mistaken for actual billing. - Add Connect's MIT attribution to the bootstrap distribution; the license text exactly matches the tagged upstream license. ## Validation (2026-10-13, Asia/Shanghai) Java: Temurin 17.0.19+10, managed with `mise`. Final fresh build: ```bash mise exec [email protected]+10 -- ./mvnw -B -nsu \ -pl shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy,shenyu-spring-boot-starter/shenyu-spring-boot-starter-plugin/shenyu-spring-boot-starter-plugin-ai-proxy \ -am clean verify \ -Dtest='*AiProxy*,OpenAi*,ConnectProtocolAdapterTest,WebClientAiProxyTransportTest,CommonAiProxyApiKeyDataSubscriberTest' \ -Dsurefire.failIfNoSpecifiedTests=false ``` - 15-module related reactor: **BUILD SUCCESS**; Checkstyle has no violations and RAT reports no unknown/unapproved licenses. - AI Proxy: **123 tests**, 0 failures/errors/skips, all 17 module test classes. Starter: **2 tests**, 0 failures/errors/skips. - Fresh JaCoCo: AI Proxy lines **713/735 (97.01%)**, instructions **3342/3486 (95.87%)**, branches **371/450 (82.44%)**. These are module numbers, not a whole-reactor coverage claim. - New `ConnectProtocolAdapter`: 100% lines/instructions, 36/38 branches (94.74%). New `OpenAiResponses`: 100% lines/instructions/branches. Starter: 100% lines/instructions. - Contracts cover identity extensions, explicit model/profile selection, SAFE failures, complete response/tool restoration, fragmented/multiline SSE, usage, stream state isolation, backpressure, cancellation, malformed/oversized input, retry/fallback and no replay after emission. - Real loopback Reactor Netty HTTP tests cover both Responses complete/stream conversions and cancellation propagating through the plugin and WebClient to the upstream socket without a successful terminal event. - `git diff --check` passes. - Existing unrelated Javadoc warnings remain; they are not new Checkstyle/RAT violations. Not executed: live-provider tests, Docker integration/E2E suites, sustained performance/load/capacity/chaos tests, a full-repository build or a distribution release. The test selector intentionally limits ancestor-module tests; it does not claim complete reactor test coverage. ### Security acceptance: draft blocker An OSV coordinate/version query of 93 external runtime dependency versions returned no known advisories for **Connect 0.1.0** or its sole runtime dependency **Gson 2.13.2** (already managed by ShenYu). This is not a security guarantee. The existing dependency graph still contains high-severity advisories, including: - `io.netty:netty-handler:4.1.111.Final`: [CVE-2025-24970 / GHSA-4g8c-wm8x-jfhw](https://github.com/advisories/GHSA-4g8c-wm8x-jfhw), patched in 4.1.118.Final. - `org.springframework.ai:spring-ai-client-chat:1.1.2`: [CVE-2026-41713 / GHSA-5852-phmh-8fhr](https://github.com/advisories/GHSA-5852-phmh-8fhr), patched in the 1.1 line at 1.1.6. Severity and affected ranges were cross-checked through GitHub's advisory API. Other pre-existing dependencies also returned advisories. Their presence does not establish exploitability in this new path. Root dependency management is unchanged; broad Netty/Spring upgrades are intentionally not mixed into this codec refactor. **The no-high-CVE dependency acceptance gate is not met**, so this PR is submitted as draft pending separate dependency remediation and maintainer review. ## Migration and rollback No database/schema migration. Existing generic Chat selectors need no changes. Responses selectors explicitly opt in, while existing DeepSeek selectors should review stricter profile validation. For rollback, restore opt-in selectors to `openai-chat` and revert the integration commit. No transport/config-sync redesign or custom replacement library is introduced. ## Contribution checklist - [x] Read the contribution guidelines. - [x] Add unit, contract and real-HTTP regression tests for the change. - [x] Pass the related-reactor `clean verify` command above. - [ ] Run the template's full-repository `./mvnw clean install -Dmaven.javadoc.skip=true` command (not claimed; related-reactor verification is recorded above). - [ ] Resolve the pre-existing high-severity dependency acceptance blocker before marking ready for merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
