biulebiu opened a new issue, #545:
URL: https://github.com/apache/rocketmq-dashboard/issues/545
## Background
The AI assistant can't reach a real LLM. The only `LlmGateway` impl is
`LlmGatewayStub`, which just streams a fake sentence and returns a stub string
for `execute`. `LlmSettings` already stores provider +
these providers (openai/anthropic/ollama/azure/bedrock) and handles the
streaming and auth differences, so we don't have to write per-provider HTTP +
SSE code ourselves. The new `LlmGateway` just picks the
`ChatModel` from config and pipes its stream into the `SseEmitter` the
frontend already reads.
Competition context:
https://www.taiyi.top/competition-details?id=2f410866-6fa3-11f1-b193-02420a000011
## Proposed scope
- Add the spring-ai modules for the providers we want
(openai/anthropic/ollama/azure/bedrock).
- New `LlmGateway` impl: read config from `LlmConfigService`, build the
matching `ChatModel` (openai-compatible ones — openai/deepseek/tongyi/ollama —
share spring-ai-openai with a custom apiBase; Claude uses
spring-ai-anthropic; azure/bedrock their own modules), call `stream()`,
and write each token to the `SseEmitter` in the format the frontend expects
(`event:message {"text":...}`, then `event:done [DONE]`).
- `execute()` uses the non-streaming `call()`.
- The new impl takes precedence over the stub; stub stays as a fallback.
- Tests with mocked ChatModels.
## Non-goals
- No tool-calling in this slice (spring-ai can do it later).
- No multi-turn history (`ChatDTO` is single-turn now).
- No config persistence (apiKey is in-memory now).
If spring-ai turns out too heavy, we can fall back to small per-family
adapters, but I'd rather not rewrite the streaming for each provider by hand.
## Compatibility and safety
- Doesn't change the `LlmGateway` interface or the SSE format the frontend
parses, so `AiController`/`AiService`/frontend stay as they are.
- Reuses `LlmConfigService`.
- Keys go only into spring-ai's auth config, not logged; apiKey is already
WRITE_ONLY in responses.
## Validation
- `mvn test` (incl. ArchUnit + new tests).
- App context starts with the new gateway winning over the stub, no bean
conflict.
- Live check: point at Ollama for an openai-compatible one, a Claude key
for Anthropic, hit `/api/ai/chat`, get a real streamed reply.
- Frontend lint/build + existing AI tests still pass.
## Related
Replaces `LlmGatewayStub` in `ops/ai`. Doesn't touch the #514 catalog
gateway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
