danny0405 commented on code in PR #9106:
URL: https://github.com/apache/hudi/pull/9106#discussion_r1252501322
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##########
@@ -91,8 +108,9 @@ public static HoodieWriteConfig createMetadataWriteConfig(
.withCleanConfig(HoodieCleanConfig.newBuilder()
.withAsyncClean(DEFAULT_METADATA_ASYNC_CLEAN)
.withAutoClean(false)
- .withCleanerParallelism(parallelism)
- .withCleanerPolicy(HoodieCleaningPolicy.KEEP_LATEST_COMMITS)
+ .withCleanerParallelism(defaultParallelism)
+ .withCleanerPolicy(HoodieCleaningPolicy.KEEP_LATEST_FILE_VERSIONS)
+ .retainFileVersions(2)
Review Comment:
Even if Uber has been running for 6+ months, it does not mean the config
work well for OSS, because while we migrating the Uber patches, many fixes and
other nuances are introduced, I would suggest we move this change to the next
release to keep the stability of existing MDT workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
