TheR1sing3un commented on code in PR #17779:
URL: https://github.com/apache/hudi/pull/17779#discussion_r2660671747
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/versioning/v2/TimelineArchiverV2.java:
##########
@@ -125,6 +123,10 @@ public int archiveIfRequired(HoodieEngineContext context,
boolean acquireLock) t
} else {
log.info("No Instants to archive");
}
+ // run compact and clean if needed even no instants were archived
+ if (!instantsToArchive.isEmpty() || config.isTimelineCompactionForced())
{
Review Comment:
> so even if we trigger compaction for commits that does not archive, it
does not take effect as expected at all.
This is related to the current strategy.
Currently, each time compact is triggered, each level will only merge **at
most one** batch of files, and then move on to the next level for merging.
So if I have 100 files at layer L0 that can be merged, but with batch=10,
that compact will only merge the first ten. Therefore, if we don't provide a
switch to force the compact to be triggered, the remaining 90 files won't be
able to be triggered for merging **until** there are some new instants comes
into archve.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
