bvaradar commented on issue #2229: URL: https://github.com/apache/hudi/issues/2229#issuecomment-724229323
Thanks @WTa-hash for the explanation. Yes, Hudi currently supports scheduling compaction based on number of elapsed commits. I have opened a jira to add configuration support for scheduling based on elapsed time period (https://issues.apache.org/jira/browse/HUDI-1381). Meanwhile, for your use case, Would it be possible to handle this inside foreatch batch logic by keeping track of time since last compaction scheduling. Can you try setting INLINE_COMPACT_NUM_DELTA_COMMITS_PROP to a higher value and reset INLINE_COMPACT_NUM_DELTA_COMMITS_PROP to 1 when you want to force a compaction to run ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
