voonhous commented on code in PR #19298:
URL: https://github.com/apache/hudi/pull/19298#discussion_r3690571913
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/query/HudiSnapshotDirectoryLister.java:
##########
@@ -54,9 +58,22 @@ public HudiSnapshotDirectoryLister(
this.lazyFileSystemView = Lazy.lazily(() -> {
HoodieTimer timer = HoodieTimer.start();
HoodieTableMetaClient metaClient = tableHandle.getMetaClient();
- HoodieTableFileSystemView fileSystemView =
getFileSystemView(lazyTableMetadata.get(), metaClient);
- if (enableMetadataTable) {
- fileSystemView.loadAllPartitions();
+ HoodieTableFileSystemView fileSystemView = null;
+ try {
+ fileSystemView = getFileSystemView(lazyTableMetadata.get(),
metaClient);
+ if (enableMetadataTable) {
+ fileSystemView.loadAllPartitions();
+ }
+ }
+ catch (Exception e) {
+ // A failure here is a metadata-table read failure (the
metastore/table itself is
+ // fine), so fall back to direct file listing instead of
failing the query.
+ if (fileSystemView != null && !fileSystemView.isClosed()) {
+ fileSystemView.close();
Review Comment:
Dropped. Left a comment on why the failed view must stay unclosed -- closing
it would take the shared HoodieTableMetadata behind lazyTableMetadata down with
it.
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/query/HudiSnapshotDirectoryLister.java:
##########
@@ -67,6 +84,22 @@ public HudiSnapshotDirectoryLister(
IndexSupportFactory.createIndexSupport(tableHandle,
lazyMetaClient, lazyTableMetadata, tableHandle.getRegularPredicates(), session)
: Optional.empty();
}
+ /**
+ * Builds a file system view that lists files directly from storage,
bypassing the metadata table.
+ * Used as the fallback when the metadata-table-backed view cannot be
loaded (e.g. an MDT read
+ * failure); it lists lazily per partition, so no {@code
loadAllPartitions()} here.
+ */
+ private static HoodieTableFileSystemView
createDirectListingFileSystemView(HoodieTableMetaClient metaClient)
Review Comment:
Done. The fallback is now a single fileListingBasedFileSystemView call with
the same completed-commits timeline HudiUtil.getFileSystemView uses.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
