sivabalan narayanan created HUDI-4021:
-----------------------------------------
Summary: Support deferring compaction when there is an inflight
delta commit
Key: HUDI-4021
URL: https://issues.apache.org/jira/browse/HUDI-4021
Project: Apache Hudi
Issue Type: Improvement
Components: writer-core
Reporter: sivabalan narayanan
in multi-writer scenario, if there is an inflight delta commit by writer1 and
if writer2 i looking to schedule compaction, the delta commit might eventually
fail. This may not be a good experience for end user. Think about regular
writes are happening via deltastreamer and a separate async process is taking
care of scheduling and executing compaction. Deltastreamer continuous mode
might get aborted whenever such overstepping happens. We should add a support
where in, compaction scheduling is deferred when an inflight delta commit is
found in the timeline.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
