Thanks Minghuang for driving Apache Hudi Catalog Support Proposal. IMO, you could use metadata synchronization tool to support Hudi Catalog based on Hudi 0.15 version, which version supports Spark 3.3, 3.4 and 3.5. Meanwhile, you can also start discussion thread about providing a unified Catalog in Apache Hudi community. How to ensure compatibility between Hudi versions also needs to be covered in the proposal.
Forward to Apache Hudi Catalog support. Regards, Nicholas Jiang On 2024/08/15 08:26:02 Minghuang Li wrote: > Hi Y’all, > > Gravitino currently supports the Apache Iceberg catalog[1] and Apache Paimon > catalog[2] for metadata management but does not support Apache Hudi[3] yet. > I'd like to start a discussion about adding Apache Hudi catalog support to > Gravitino. > > This integration could potentially expand Gravitino's capabilities in > managing data lake metadata. > > I've opened a Github epic issue to track this proposal: > #4306<https://github.com/apache/gravitino/issues/4306> > > A draft design document is available here (will also be posted to issue): > <https://docs.google.com/document/d/1kJN4JQaQ_-_rdk9jR4fiL-LUxqRaSG00q9cSmqWBNR8/edit?usp=sharing> > > I'd appreciate the community's thoughts on: > 1. Is this feature aligned with Gravitino's goals? > 2. Are there any concerns or potential issues we should address? > 3. Any suggestions for improving the design or implementation approach? > > If there's interest in pursuing this, I'm willing to move forward as outlined > in the timeline. However, I'm open to any feedback or alternative approaches > the community might suggest. > > Thanks for your time and input. > > Best regards, > Minghuang Li > > [1] > https://github.com/apache/gravitino/blob/main/docs/lakehouse-iceberg-catalog.md > [2] > https://github.com/apache/gravitino/blob/main/docs/lakehouse-paimon-catalog.md > [3] https://hudi.apache.org >
