[
https://issues.apache.org/jira/browse/HUDI-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kong Wei closed HUDI-6714.
--------------------------
Resolution: Won't Do
already has this parameter to enable such feature
hoodie.compact.schedule.inline
> HoodieStreamer support only schedule the compaction plan but not execute the
> plan
> ---------------------------------------------------------------------------------
>
> Key: HUDI-6714
> URL: https://issues.apache.org/jira/browse/HUDI-6714
> Project: Apache Hudi
> Issue Type: New Feature
> Reporter: Kong Wei
> Assignee: Kong Wei
> Priority: Major
>
> For HoodieStreamer(aka HoodieDeltaStreamer) writing MOR table, the compaction
> mode can be *async.*
> In the async compaction mode, the hoodie-streamer will schedule one
> compaction plan after each write operation and execute compaction plan if
> need. But the execution of compaction will share the spark job resource,
> which may cause the write delay.
> In our cases, we want to execute the compaction offline to save the spark
> resource of streamer and reduce the write latency. And we found that
> scheduling the compaction plan offline will fail while streamer is writing
> (means we have to stop the streamer in order to schedule the plan offline).
> So we want the streamer only to schedule the compaction plan but not to
> execute it.
> But currently the streamer seems not support such case. If we set the
> `--disable-compaction` to false, the streamer will not schedule the
> compaction plan anymore.
> So I want to add a param named --{_}enable-schedule-compaction{_} in the
> streamer,
> and we can set --{_}disable-compaction{_}=false and
> {_}enable-schedule-compaction{_}=true to enable only schedule the compaction
> in streamer.
> the cases like below:
> ||--disable-compaction||--enable-schedule-compaction||schedule plan||execute
> plan||
> |true|true or false|true|true|
> |false|true|true|false|
> |false|false|false|false|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
