nsivabalan commented on code in PR #17935:
URL: https://github.com/apache/hudi/pull/17935#discussion_r2720292677
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##########
@@ -159,22 +159,22 @@ public static HoodieWriteConfig createMetadataWriteConfig(
final long maxLogFileSizeBytes =
writeConfig.getMetadataConfig().getMaxLogFileSize();
// Borrow the cleaner policy from the main table and adjust the cleaner
policy based on the main table's cleaner policy
- HoodieCleaningPolicy dataTableCleaningPolicy =
writeConfig.getCleanerPolicy();
+ HoodieCleaningPolicy metadataTableCleaningPolicy =
writeConfig.getMetadataConfig().getCleanerPolicy();
HoodieCleanConfig.Builder cleanConfigBuilder =
HoodieCleanConfig.newBuilder()
.withAsyncClean(DEFAULT_METADATA_ASYNC_CLEAN)
.withAutoClean(false)
.withCleanerParallelism(MDT_DEFAULT_PARALLELISM)
.withFailedWritesCleaningPolicy(failedWritesCleaningPolicy)
.withMaxCommitsBeforeCleaning(writeConfig.getCleaningMaxCommits())
- .withCleanerPolicy(dataTableCleaningPolicy);
+ .withCleanerPolicy(metadataTableCleaningPolicy);
- if
(HoodieCleaningPolicy.KEEP_LATEST_COMMITS.equals(dataTableCleaningPolicy)) {
+ if
(HoodieCleaningPolicy.KEEP_LATEST_COMMITS.equals(metadataTableCleaningPolicy)) {
Review Comment:
I am not sure if 1.2 X is needed anymore.
previously, there is no way one can configure cleaner configuration for mdt,
and hence we were deriving from data table's clean policy and hence added the
20% buffer.
but if we are looking to add explicit configurations, we don't need buffer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
