garyli1019 commented on code in PR #6025:
URL: https://github.com/apache/hudi/pull/6025#discussion_r913431715
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -507,6 +507,20 @@ private FlinkOptions() {
.defaultValue(4) // default WRITE_TASKS * COMPACTION_DELTA_COMMITS * 0.2
(assumes 5 commits generate one bucket)
.withDescription("Parallelism of tasks that do actual compaction,
default is 4");
+ public static final ConfigOption<String> COMPACTION_SEQUENCE = ConfigOptions
+ .key("compaction.sequence")
+ .stringType()
+ .defaultValue("LIFO") // default WRITE_TASKS * COMPACTION_DELTA_COMMITS
* 0.2 (assumes 5 commits generate one bucket)
Review Comment:
LIFO is not recommended. It's possible the oldest plan never get executed
and the active timeline get longer and longer.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/HoodieFlinkCompactor.java:
##########
@@ -99,6 +94,12 @@ public void start(boolean serviceMode) throws Exception {
}
}
+ public void shutDown() throws Exception {
+ if (compactionScheduleService != null) {
Review Comment:
didn't check this code for a while. Did we support async compaction
scheduling already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
