ArafatKhan2198 opened a new pull request, #10805: URL: https://github.com/apache/ozone/pull/10805
## What changes were proposed in this pull request? This PR adds a dedicated `gateway` LLM provider to the Recon chatbot so operators can route all chatbot traffic through a single OpenAI-compatible gateway (for example LiteLLM), instead of misconfiguring direct provider keys. **Problem** When customers front multiple LLM vendors (OpenAI, Gemini, Anthropic) with one OpenAI-compatible gateway, the existing chatbot configuration is awkward: they must set `provider=openai` and list non-OpenAI model aliases under `openai.models`. That is confusing, easy to misconfigure, and does not reflect how the gateway is actually used. **Solution** This PR introduces a new `gateway` provider that uses the same OpenAI-compatible LangChain4j client under the hood, with explicit configuration keys: - `ozone.recon.chatbot.provider=gateway` - `ozone.recon.chatbot.gateway.api.key` - `ozone.recon.chatbot.gateway.base.url` (required; no default) - `ozone.recon.chatbot.gateway.models` (comma-separated model aliases) When `gateway.api.key` is set, Recon registers the gateway provider and exposes the configured model list via the existing chatbot models API. If `gateway.base.url` is missing, Recon fails fast with a clear error rather than silently falling back to the public OpenAI endpoint. Existing `openai`, `gemini`, and `anthropic` providers are unchanged. **Implementation notes** - New config constants in `ChatbotConfigKeys.java` - `LangChain4jDispatcher` registers `gateway` at startup and routes requests through `buildGatewayModel()` - Unit tests cover availability, model listing, routing, and the missing-base-URL validation path - A commented gateway example block is added to the compose `docker-config` for local testing Model names must match the gateway’s configured aliases exactly. There is no auto-discovery from `/v1/models`; operators maintain the list in `gateway.models`. ### Administrator guide updates for the gateway provider are planned separately. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-15906 ## How was this patch tested? - Unit tests: - `mvn -pl :ozone-recon test -Dtest=TestLangChain4jDispatcher,TestLlmRouting -DskipShade -DskipRecon -DskipDocs` - Manual smoke test on a local Docker compose cluster: - Enabled the chatbot with `provider=gateway` - Pointed `gateway.base.url` at a LiteLLM OpenAI-compatible endpoint - Configured `gateway.models` with aliases for OpenAI, Gemini, and Anthropic models exposed by the gateway - Verified chatbot requests succeed end-to-end through the gateway for multiple model aliases <img width="592" height="296" alt="Screenshot 2026-07-18 at 9 26 10 PM" src="https://github.com/user-attachments/assets/828dda67-fb02-4155-93a8-4d88425a36b6"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
