danny0405 commented on code in PR #19949:
URL: https://github.com/apache/hudi/pull/19949#discussion_r4011240742
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/IncrementalInputSplits.java:
##########
@@ -447,9 +447,10 @@ private List<MergeOnReadInputSplit> getInputSplits(
.filter(logPath ->
!logPath.endsWith(HoodieCDCUtils.CDC_LOGFILE_SUFFIX))
.collect(Collectors.toList()));
String basePath =
fileSlice.getBaseFile().map(BaseFile::getPath).orElse(null);
- // the latest commit is used as the limit of the log reader instant
upper threshold,
- // it must be at least the latest instant time of the file slice to
avoid data loss.
- String latestCommit =
InstantComparison.minInstant(fileSlice.getLatestInstantTime(), endInstant);
+ // The latest commit is the physical upper threshold of the log reader.
It must cover
+ // both the selected file slice and the query end to avoid data loss.
The instant range
+ // remains the logical query boundary and filters out records beyond the
query end.
+ String latestCommit =
InstantComparison.maxInstant(fileSlice.getLatestInstantTime(), endInstant);
Review Comment:
seems not right, the endInstant is the forced right boundary so that no file
slice instant time can overstrip?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
