bvaradar commented on issue #2151: URL: https://github.com/apache/hudi/issues/2151#issuecomment-706987817
@tandonraghav : It is by design that a "file" which is pending compaction is not scheduled for compaction till the compaction is done. One another knob is the strategy for selecting files for compaction Scheduling which is also pluggable. For example : You can implement your own CompactionStrategy to prioritize files belonging to "hot" partitions and keeping the number of files per compaction to be less. Basically, You need to run compactions at higher frequencies and keeping delta.commits=1 if you are trying to optimize for data-freshness but want to use Read-Optimized queries. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
