vinothchandar opened a new issue, #19256: URL: https://github.com/apache/hudi/issues/19256
### Feature Description **What the feature achieves:** Every Hudi table becomes something you can talk to. This feature adds an optional, self-hostable serving layer to the project: a set of Helm charts (`hudi-lakehouse`) that deploy Trino with the Hudi connector alongside a new `hudi-agent-gateway` service — a single process that hosts an LLM agent loop with guarded, read-only SQL tools over the lakehouse, exposes the same tools to external agents via MCP, and ships a first-party chat UI. Users ask questions in natural language; the agent inspects the catalog, writes SQL, runs it through Trino, and answers grounded in real query results. **Why this feature is needed:** Hudi has always been more than a table format — self-managing table services, ingestion utilities, catalog syncing and a CLI make it a data platform. AI-assisted access and operations are the natural next surface of that platform, and there is real value in the project shipping it first-party rather than leaving every team to assemble and glue themselves: the tools stay engine-native (catalog introspection, guarded SQL, and later table-services operations), the whole stack is open and self-hostable with local inference (Ollama) as the out-of-box default and pluggable model providers (vLLM, OpenAI, Anthropic), and answer quality can build on Hudi-specific knowledge no generic text-to-SQL layer has. The longer arc is agents that not only answer questions but help operate tables — tuning, inspecting, and maintaining them. This issue is the umbrella; the linked sub-issues break the work down, and an initial implementation is ready to land as the first PR. ### User Experience **How users will use this feature:** - `helm install` the `hudi-lakehouse` charts against an existing metastore and object store (or run the included one-command minikube environment); the gateway comes up next to Trino. - Chat UI at the gateway's `/ui/` for interactive analysis and built-in chat; `POST /v1/chat` (JSON or SSE streaming) for programmatic use; `/mcp` so tools like Claude can operate on the lakehouse directly; a Connect panel surfaces every endpoint. - Inference provider is a values-file choice: Ollama by default, Anthropic or OpenAI via an API key Secret, vLLM as an optional part of the same install, or any OpenAI-compatible URL. The UI adapts to whatever the cluster runs. - All agent SQL is guarded: parsed, read-only-verified, row-capped, and time-limited; every tool invocation is logged. - No changes to existing Hudi APIs, configs, or storage — everything is new, optional, and additive. ### Hudi RFC Requirements **RFC PR link:** N/A **Why RFC is/isn't needed:** - Does this change public interfaces/APIs? No - Does this change storage format? No - Justification: purely additive, optional modules (charts + a Python service); nothing existing changes behavior. Direction and design are open for community input on a GitHub Discussion to follow (mirrored to dev@). Sub-efforts that do touch engine or format surfaces (e.g. vector search through the SQL endpoint) will get their own RFC/design before work starts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
