cshuo commented on code in PR #19984:
URL: https://github.com/apache/hudi/pull/19984#discussion_r4043567320
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/lsm/HoodieLsmFileGroupReader.java:
##########
@@ -142,10 +145,14 @@ private HoodieLsmFileGroupReader(
.sortOutputs(false)
.inflightInstantsAllowed(allowInflightInstants)
.build();
- // filter log files by instant range.
- if (logFiles != null && readerContext.getInstantRange().isPresent()) {
- InstantRange instantRange = readerContext.getInstantRange().get();
- logFiles = logFiles.filter(logFile ->
instantRange.isInRange(logFile.getDeltaCommitTime()));
Review Comment:
The existing filter works when InstantRange is present, but Spark
time-travel queries leave it empty and pass the as-of timestamp through
latestCommitTime. A selected slice can still contain later logs, so we need
this cutoff.
Replacing latestCommitTime with InstantRange in the reader context could be
handled in a follow-up refactoring PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
