yihua commented on code in PR #11947:
URL: https://github.com/apache/hudi/pull/11947#discussion_r1797776979
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java:
##########
@@ -308,6 +309,38 @@ public static HoodieTimeline getCommitsTimelineAfter(
return timelineSinceLastSync;
}
+ public static HoodieTimeline getCommitsTimeLineAfterByCompletionTimeRange(
Review Comment:
If the completion time to resume from for a incremental query corresponds to
an instant in the archived timeline, we should fallback to the snapshot query
instead of pulling the archived timeline, as reading archived timeline can be
expensive.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java:
##########
@@ -532,6 +532,13 @@ public boolean isBeforeTimelineStarts(String instant) {
&& compareTimestamps(instant, LESSER_THAN,
firstNonSavepointCommit.get().getTimestamp());
}
+ @Override
+ public boolean isBeforeTimelineStartsByCompletionTime(String completionTime)
{
+ Option<HoodieInstant> firstNonSavepointCommit =
getFirstNonSavepointCommit();
Review Comment:
`getInstantsOrderedByCompletionTime()` would be useful.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstantTimeGenerator.java:
##########
@@ -140,6 +140,10 @@ public static String fixInstantTimeCompatibility(String
instantTime) {
}
}
+ public static String getStrictlyLowerTimestamp(String instantTime) {
+ return instantTimeMinusMillis(instantTime, 1);
+ }
Review Comment:
If this method is not widely used, IMO we should directly use
`instantTimeMinusMillis(instantTime, 1)` for clarity instead of introducing a
new utility method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
