GitHub user jianlirong added a comment to the discussion: [Proposal] Iceberg subsystem for datalake_fdw — design proposal
Here I want to add a little bit about why I think Java is the right choice for a unification/universal iceberg catalog proxy. When building a proxy layer that needs to interoperate with the broad ecosystem of Apache Iceberg catalog services, Java is the clear implementation choice over C/C++. The primary reason is native client SDK coverage. Every major catalog service in the Iceberg ecosystem — including Apache Hive Metastore, AWS Glue Data Catalog, Project Nessie, Apache Polaris, Apache Gravitino, Databricks Unity Catalog, and JDBC-based catalogs — provides a first-class, officially maintained Java client SDK. The Apache Iceberg project itself is Java-native, and its official library (iceberg-java) ships dedicated integration modules for each of these catalogs out of the box: iceberg-hive-metastore, iceberg-aws (for Glue), iceberg-jdbc, and a built-in REST catalog client that covers any catalog implementing the Iceberg REST specification. C/C++ support, by contrast, is significantly more limited. The apache/iceberg-cpp library currently only provides a native client for the Iceberg REST Catalog protocol. This means it can reach catalogs that have adopted the REST specification — such as Polaris, Unity Catalog, and Gravitino — but it has no native support for Hive Metastore's Thrift protocol, JDBC-based catalogs, or the Hadoop catalog, all of which require JVM-based clients. There is no official C++ SDK for AWS Glue at all. Legacy and widely-deployed catalogs like Hive Metastore remain inaccessible from C++ without resorting to unmaintained third-party libraries. In short, a Java-based proxy can speak natively to every catalog in the market today using well-maintained, officially supported SDKs. A C/C++ proxy would be fundamentally incomplete, requiring workarounds or re-implementations for a significant portion of the catalog landscape. For a proxy whose core value is broad catalog compatibility, Java is the only pragmatic foundation. GitHub link: https://github.com/apache/cloudberry/discussions/1683#discussioncomment-17062626 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
