prasannarajaperumal commented on code in PR #5958:
URL: https://github.com/apache/hudi/pull/5958#discussion_r923158235
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java:
##########
@@ -266,6 +272,20 @@ public class HoodieCompactionConfig extends HoodieConfig {
.sinceVersion("0.11.0")
.withDocumentation("When rewriting data, preserves existing
hoodie_commit_time");
+ public static final ConfigProperty<String> PARTITIONS_FOR_COMPACTION =
ConfigProperty
Review Comment:
Sorry I dont follow why we need a list of partitions for compactions as
config?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -362,7 +363,7 @@ public void bootstrap(Option<Map<String, String>>
extraMetadata) {
protected void rollbackFailedBootstrap() {
LOG.info("Rolling back pending bootstrap if present");
HoodieTable<T, I, K, O> table = createTable(config, hadoopConf);
- HoodieTimeline inflightTimeline =
table.getMetaClient().getCommitsTimeline().filterPendingExcludingCompaction();
+ HoodieTimeline inflightTimeline =
table.getMetaClient().getCommitsTimeline().filterPendingExcludingMajorAndMinorCompaction();
Review Comment:
I believe you are using the term log compaction? Why refer to it here as
minor compaction? (We should stick to one name everywhere)
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java:
##########
@@ -314,6 +334,11 @@ public class HoodieCompactionConfig extends HoodieConfig {
.withDocumentation("When enable, hoodie will auto merge several small
archive files into larger one. It's"
+ " useful when storage scheme doesn't support append operation.");
+ public static final ConfigProperty<String> LOG_COMPACTION_BLOCKS_THRESHOLD =
ConfigProperty
+ .key("hoodie.log.compaction.blocks.threshold")
+ .defaultValue("5")
Review Comment:
Is there a size threshold to exclude big log blocks? Why do log compaction
on log blocks that are big enough? Thoughts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
