fengjian428 commented on code in PR #6144:
URL: https://github.com/apache/hudi/pull/6144#discussion_r937320646
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/ScheduleCompactionActionExecutor.java:
##########
@@ -140,6 +140,17 @@ private Option<Pair<Integer, String>>
getLatestDeltaCommitInfo() {
return Option.empty();
}
+ private Option<Pair<Integer, String>>
getLatestDeltaCommitInfoSinceLastCompactionRequest() {
+ Option<Pair<HoodieTimeline, HoodieInstant>> deltaCommitsInfo =
+
CompactionUtils.getDeltaCommitsSinceLatestCompactionRequest(table.getActiveTimeline());
+ if (deltaCommitsInfo.isPresent()) {
+ return Option.of(Pair.of(
+ deltaCommitsInfo.get().getLeft().countInstants(),
Review Comment:
For now, NUM_COMMITS or TIME_ELAPSED compaction trigger strategy will check
the number or time of delta-commits after the last successful compaction. so if
the offline compaction application crash for a while(or async compaction is
very slow), there will be a lot of compaction request（one request per delta
commit） in the timeline, and that will have a side effect on performance.
so this PR provides a new strategy not to check the last successful
compaction but check the last compaction request if possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
