nsivabalan commented on code in PR #8422:
URL: https://github.com/apache/hudi/pull/8422#discussion_r1162197182
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -3030,24 +3030,26 @@ private void validate() {
.equals(HoodieFailedWritesCleaningPolicy.EAGER.name()), "To enable
optimistic concurrency control, set hoodie.cleaner.policy.failed.writes=LAZY");
}
- HoodieCleaningPolicy.valueOf(writeConfig.getString(CLEANER_POLICY));
- // Ensure minInstantsToKeep > cleanerCommitsRetained, otherwise we will
archive some
- // commit instant on timeline, that still has not been cleaned. Could
miss some data via incr pull
- int minInstantsToKeep =
Integer.parseInt(writeConfig.getStringOrDefault(HoodieArchivalConfig.MIN_COMMITS_TO_KEEP));
- int maxInstantsToKeep =
Integer.parseInt(writeConfig.getStringOrDefault(HoodieArchivalConfig.MAX_COMMITS_TO_KEEP));
- int cleanerCommitsRetained =
-
Integer.parseInt(writeConfig.getStringOrDefault(HoodieCleanConfig.CLEANER_COMMITS_RETAINED));
- checkArgument(maxInstantsToKeep > minInstantsToKeep,
- String.format(
- "Increase %s=%d to be greater than %s=%d.",
- HoodieArchivalConfig.MAX_COMMITS_TO_KEEP.key(),
maxInstantsToKeep,
- HoodieArchivalConfig.MIN_COMMITS_TO_KEEP.key(),
minInstantsToKeep));
- checkArgument(minInstantsToKeep > cleanerCommitsRetained,
- String.format(
- "Increase %s=%d to be greater than %s=%d. Otherwise, there is
risk of incremental pull "
- + "missing data from few instants.",
- HoodieArchivalConfig.MIN_COMMITS_TO_KEEP.key(),
minInstantsToKeep,
- HoodieCleanConfig.CLEANER_COMMITS_RETAINED.key(),
cleanerCommitsRetained));
+ HoodieCleaningPolicy cleaningPolicy =
HoodieCleaningPolicy.valueOf(writeConfig.getString(CLEANER_POLICY));
+ if (cleaningPolicy == HoodieCleaningPolicy.KEEP_LATEST_COMMITS) {
Review Comment:
NTR(note to reviewer) : no changes except that this validation is done only
incase of KEEP LATEST COMMITS cleaner policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
