tju-yxq opened a new pull request, #2209: URL: https://github.com/apache/rocketmq-dashboard/pull/2209
Closes #2208 ## What changed AI chat work now follows the lifecycle of the downstream SSE response instead of continuing independently until the provider timeout. - Replaced `CallerRunsPolicy` with explicit rejection. A saturated chat executor no longer runs an HTTP/CLI provider call on the servlet request thread; the returned emitter is completed immediately with `503 / llm.gateway.overloaded` and a retry hint. - Added `LlmSseSession`, which owns one emitter and one submitted `Future`, registers completion/timeout/error callbacks, handles termination-before-attachment races, and guarantees a single terminal transition. - Connected client completion, response timeout, transport errors, and gateway shutdown to task cancellation. - Suppressed provider-error logging and secondary SSE writes after downstream cancellation. - Kept incomplete-config and unsupported-provider errors independent of the bounded executor, so those small responses still work while chat capacity is full. - Tracked active sessions during shutdown and cancelled them before stopping the executor. - Destroyed the Claude streaming child process when its worker is interrupted, matching the existing non-streaming CLI cleanup behavior. The production executor remains bounded at 16 concurrent chat tasks with no queue. Existing 60-second HTTP and 300-second CLI emitter/provider timeouts remain upper bounds; disconnect cancellation can now release capacity earlier. ## Why explicit rejection is needed With the previous `SynchronousQueue` plus `CallerRunsPolicy`, the seventeenth chat ran synchronously inside `chat()` while sixteen workers were blocked. A deterministic regression test now holds the only injected test worker, submits another request, and proves that the second call returns its emitter promptly, invokes no second provider operation, and contains the structured overload event. ## Verification Run from `server` on commit `66661294`: ```text mvn -DskipTests=false "-Dtest=LlmSseSessionTest,OpenAiCompatibleLlmGatewayTest,ClaudeCodeAgentProviderTest,StudioApplicationTest" test Tests run: 16, Failures: 0, Errors: 0, Skipped: 0 BUILD SUCCESS mvn -DskipTests=false test Tests run: 1205, Failures: 0, Errors: 0, Skipped: 0 BUILD SUCCESS ``` Additional checks: ```text Checkstyle violations: 0 git diff --check: passed ``` The tests use controlled executors, latches, emitters, futures, and a mocked CLI process to cover overload rejection, completion/timeout/error cancellation, late task attachment, HTTP interruption, single terminal sequences, shutdown cancellation, and Claude process destruction. No live LLM provider, Claude/Qoder installation, browser disconnect, or deployed server was used. ## Related changes Issue #1564 concerns the upstream OpenAI-compatible response read deadline. PR #2042 bounds Claude streaming output. PR #2197 handles provider error envelopes. PR #2207 isolates CLI subprocess environments. This PR does not replace those contracts; its overlap with the Claude provider is limited to interrupted-process cleanup. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
