danny0405 commented on code in PR #19041:
URL: https://github.com/apache/hudi/pull/19041#discussion_r3601326718
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/TimelineArchiverV2.java:
##########
@@ -234,7 +235,11 @@ private List<HoodieInstant> getCommitInstantsToArchive()
throws IOException {
config.getCleanerPolicy());
earliestInstantToRetainCandidates.add(earliestInstantToRetainForClustering);
- // 4. If metadata table is enabled, do not archive instants which are more
recent than the last compaction on the
+ // 4. If enabled, block archival based on ECTR from the last completed
clean to ensure we don't archive
+ // commits that have data files that haven't been cleaned yet.
+
earliestInstantToRetainCandidates.add(getEarliestInstantToRetainForClean(table,
completedCommitsTimeline, config));
Review Comment:
this function seems to have some overlapping with the clustering clean
retain instant blocking logic, see `getEarliestInstantToRetainForClustering`,
is there any other actions(or instant type) that could incur data correctness
issue for archiving non-cleaned commits?(the clustering is already handled
appropriately).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
