yihua commented on code in PR #18337:
URL: https://github.com/apache/hudi/pull/18337#discussion_r3095459325
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCleanConfig.java:
##########
@@ -426,6 +437,11 @@ public HoodieCleanConfig.Builder
withMaxCommitsToClean(long maxCommitsToClean) {
return this;
}
+ public HoodieCleanConfig.Builder withMaxDurationToCreateEmptyClean(long
duration) {
Review Comment:
🤖 nit: could you rename the parameter to `durationMs` to clarify the unit
matches the config key `MAX_DURATION_TO_CREATE_EMPTY_CLEAN_MS`? This would
align with similar methods like `cleanerNumHoursRetained`.
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify
any suggestions before applying.</i></sub>
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCleanConfig.java:
##########
@@ -250,6 +251,16 @@ public class HoodieCleanConfig extends HoodieConfig {
.markAdvanced()
.withDocumentation("Maximum number of commits to clean in one clean
commit. Applicable only when the clean policy is based on KEEP_LATEST_COMMITS
or KEEP_LATEST_HOURS");
+ public static final ConfigProperty<Long>
MAX_DURATION_TO_CREATE_EMPTY_CLEAN_MS = ConfigProperty
+ .key("hoodie.write.empty.clean.create.duration.ms")
+ .defaultValue(-1L)
+ .markAdvanced()
+ .withDocumentation("In some cases empty clean commit needs to be created
to ensure the clean planner "
+ + "does not look through entire dataset if there are no clean plans.
This is possible for append-only "
+ + "dataset. Also, for these datasets we cannot ignore clean
completely since in the future there could "
+ + "be upsert or replace operations. By creating empty clean commit,
earliest_commit_to_retain value "
+ + "will be updated so that now clean planner can only check for
partitions that are modified after the "
Review Comment:
🤖 nit: small typo in the documentation string — 'there by' should be
'thereby'.
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify
any suggestions before applying.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
