kamronis commented on code in PR #17607:
URL: https://github.com/apache/hudi/pull/17607#discussion_r2630107378
##########
hudi-common/src/main/java/org/apache/hudi/common/table/cdc/HoodieCDCExtractor.java:
##########
@@ -341,6 +341,20 @@ private Option<FileSlice> getDependentFileSliceForLogFile(
.filter(logFile -> !logFile.equals(currentLogFileName))
.map(logFile -> new StoragePath(partitionPath, logFile))
.collect(Collectors.toList());
+ // get files list from unfinished compaction commit
+ List<StoragePath> filesToCompact =
metaClient.getActiveTimeline().getInstants().stream().filter(
+ i -> i.compareTo(instant) < 0 && !i.isCompleted() &&
i.getAction()
+ .equals(HoodieActiveTimeline.COMPACTION_ACTION))
Review Comment:
I looked through the code and tested again. I found that I was wrong,
BaseFile is really set as dependent file for flieslice.
My tests worked before without addition of BaseFile, because CDC file was
added from compaction commit and data was read from CDC file.
I reworked my code to add BaseFile from compaction commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
