bvaradar commented on code in PR #8378:
URL: https://github.com/apache/hudi/pull/8378#discussion_r1162331166
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java:
##########
@@ -320,11 +326,9 @@ public void refreshTimeline() throws IOException {
.build();
switch (meta.getTableType()) {
case COPY_ON_WRITE:
- this.commitTimelineOpt =
Option.of(meta.getActiveTimeline().getCommitTimeline().filterCompletedInstants());
- this.allCommitsTimelineOpt =
Option.of(meta.getActiveTimeline().getAllCommitsTimeline());
- break;
case MERGE_ON_READ:
- this.commitTimelineOpt =
Option.of(meta.getActiveTimeline().getDeltaCommitTimeline().filterCompletedInstants());
+ this.commitTimelineOpt =
Option.of(meta.getActiveTimeline().getCommitTimeline().filterCompletedInstants());
+ this.deltaCommitTimelineOpt =
Option.of(meta.getActiveTimeline().getDeltaCommitTimeline().filterCompletedInstants());
Review Comment:
Instead of adding extra timelines, we should just use getCommitsTimeline for
both COW and MOR. In checkpoint selection, For MOR, if there are no
deltacommits, use latest .commit, else filter out all .commits and pick the
latest .deltacommit
this.commitTimelineOpt =
Option.of(meta.getActiveTimeline().getCommitsTimeline().filterCompletedInstants());
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
