Shizhi Chen created HUDI-4329:
---------------------------------
Summary: Add separate control for compaction operation sync/async
mode
Key: HUDI-4329
URL: https://issues.apache.org/jira/browse/HUDI-4329
Project: Apache Hudi
Issue Type: Improvement
Components: flink
Reporter: Shizhi Chen
Assignee: Shizhi Chen
*Problem Review*
The compact operation sync/async in CompactionFunction is now controlled by
FlinkOptions#COMPACTION_ASYNC_ENABLED
{code:java}
public CompactFunction(Configuration conf) {
this.conf = conf;
this.asyncCompaction = StreamerUtil.needsAsyncCompaction(conf);
}
{code}
While in fact it cannot be switched to sync mode because the pipeline defined
by sync compaction will only include the clean but not compact operators.
{code:java}
// compaction
if (StreamerUtil.needsAsyncCompaction(conf)) {
return Pipelines.compact(conf, pipeline);
} else {
return Pipelines.clean(conf, pipeline);
}
{code}
*Improvement*
Add another separate control switch for compaction operation sync/async mode.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
