danny0405 commented on code in PR #10053:
URL: https://github.com/apache/hudi/pull/10053#discussion_r1390137560
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1000,7 +1000,9 @@ private static List<FileSlice>
getPartitionFileSlices(HoodieTableMetaClient meta
if (mergeFileSlices) {
if
(metaClient.getActiveTimeline().filterCompletedInstants().lastInstant().isPresent())
{
fileSliceStream = fsView.getLatestMergedFileSlicesBeforeOrOn(
- partition,
metaClient.getActiveTimeline().filterCompletedInstants().lastInstant().get().getTimestamp());
+ // including pending compaction instant as the last instant so
that the finished delta commits
+ // that start earlier than the compaction can be queried.
+ partition,
metaClient.getActiveTimeline().filterCompletedAndCompactionInstants().lastInstant().get().getTimestamp());
Review Comment:
yeah, just additionally including the pending compaction instants because
now we do file slicing based on completion time, we should use compaction
instant to filter out the file slices if there are del_commit with smaller
start time but greater completion time (than the compaction)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
