codope commented on code in PR #10915:
URL: https://github.com/apache/hudi/pull/10915#discussion_r1549072910
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java:
##########
@@ -147,6 +147,12 @@ public HoodieTimeline
filterCompletedAndCompactionInstants() {
|| s.getAction().equals(HoodieTimeline.COMPACTION_ACTION)),
details);
}
+ @Override
+ public HoodieTimeline filterCompletedCommitInstants() {
Review Comment:
Let's unit test this.
##########
hudi-common/src/main/java/org/apache/hudi/common/util/CompactionUtils.java:
##########
@@ -285,8 +285,7 @@ public static List<HoodieInstant>
getPendingCompactionInstantTimes(HoodieTableMe
*/
public static Option<Pair<HoodieTimeline, HoodieInstant>>
getDeltaCommitsSinceLatestCompaction(
HoodieActiveTimeline activeTimeline) {
- Option<HoodieInstant> lastCompaction = activeTimeline.getCommitTimeline()
- .filterCompletedInstants().lastInstant();
+ Option<HoodieInstant> lastCompaction =
activeTimeline.filterCompletedCommitInstants().lastInstant();
Review Comment:
This change is causing a test failure -
`TestCDCDataFrameSuite.testMORDataSourceWrite`. Can you please check that?
Also, I think instead of adding yet another API, we can clean up the
existing ones.
For instance, have separate method for each action - `get<ACTION>Timeline`.
So, `getCommitTimeline` only checks for COMMIT action.
##########
hudi-common/src/test/java/org/apache/hudi/common/util/TestCompactionUtils.java:
##########
@@ -272,6 +272,59 @@ public void
testGetDeltaCommitsSinceLatestCompaction(boolean hasCompletedCompact
}
}
+ @Test
+ public void
testGetDeltaCommitsSinceLastCompactionWithCompletedReplaceCommits() {
Review Comment:
+1 for the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
