cshuo commented on code in PR #19373:
URL: https://github.com/apache/hudi/pull/19373#discussion_r3656775808
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -571,9 +573,10 @@ private ClosableIterator<IndexedRecord>
readSliceWithFilter(Predicate predicate,
.explicitInstants(validInstantTimestamps).build());
// If reuse is enabled and full scan is allowed for the partition, we can
reuse the file readers for base files and the reader context for the log files.
+ boolean shouldReuse = reuse &&
isFullScanAllowedForPartition(fileSlice.getPartitionPath());
+ boolean useLsmReader = !shouldReuse &&
shouldUseLsmReader(metadataMetaClient, fileSlice);
Review Comment:
MDT uses HFile as its fixed base-file format, and native logs are written
using the table's base-file format, so MDT native logs are HFile as well.
`HoodieAvroReaderContext#getFileRecordIterator` applies the key/prefix filter
through `HoodieNativeAvroHFileReader`, which supports both predicates.
Therefore, the LSM log iterators are already filtered for this MDT path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
