voonhous opened a new issue, #20002: URL: https://github.com/apache/hudi/issues/20002
Follow-up to #18780 (RFC-105 hudi-trino migration), raised in review of trinodb/trino#30682. ### What Trino's lakehouse connector reuses the Hudi connector's classes but cannot install `HudiModule`, so `LakehouseHudiModule` has to provide the bindings `HudiSplitManager` needs itself. One of them is the metastore getter, `BiFunction<ConnectorIdentity, HiveTransactionHandle, HiveMetastore>`. `HudiModule` implements it as `transactionManager.get(transactionHandle, identity).getMetastore()`, but `HudiMetadata.getMetastore()` is package-private, so code outside `io.trino.plugin.hudi` cannot do the same. The lakehouse connector currently falls back to `HiveMetastoreFactory.createMetastore(Optional.of(identity))`. That creates a new metastore for every `getSplits` call and skips the transaction-scoped cached one, so `hudi.per-transaction-metastore-cache-maximum-size` has no effect in lakehouse catalogs. ### Proposed fix Either of: - Make `HudiMetadata.getMetastore()` public. - Expose a reusable bindings module (for example the executors and the metastore getter from `HudiModule`) that the lakehouse connector can install, so it stops re-implementing `HudiModule` on every release. ### Why here Under RFC-105, `HudiMetadata` and `HudiModule` live in `org.apache.hudi:hudi-trino`, so the Trino side cannot change their visibility. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
