cshuo opened a new issue, #20231: URL: https://github.com/apache/hudi/issues/20231
### Task Description Add an optional, persistent Spark Connect endpoint under `hudi-lakehouse`, following the direction in [discussion #19843](https://github.com/apache/hudi/discussions/19843). The endpoint will serve Hudi reads and writes through ADBC clients and be shared by the agent gateway, Superset, and Zeppelin. ### Scope - Provide a Hudi-enabled Spark Connect image with pinned, compatible Spark/Hudi/Scala/Java versions and required dependencies. - Add a `hudi-spark-connect` Helm chart with catalog, object storage, existing Secret, resource, and service configuration. Default to cluster-internal access. - Integrate with the local-dev quickstart without breaking the existing Trino and Spark writer paths. - Install ADBC dependencies on clients; the server runs Spark Connect, not a separate ADBC service. ### Acceptance Criteria - [ ] The image builds reproducibly and the chart passes Helm lint/template checks. - [ ] A clean local Kubernetes environment can deploy the endpoint against the local-dev catalog and object store. - [ ] A Foundry ADBC client can create a Hudi table and execute SELECT, INSERT, UPDATE, and DELETE, with readback assertions. - [ ] Tables remain accessible after an endpoint restart. - [ ] Startup, connection, and teardown instructions are included. Gateway write tools are out of scope; endpoint write access is for direct clients. ### Task Type Other — runtime packaging and deployment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
