danny0405 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396767580
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##########
@@ -111,13 +114,13 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
};
this.timelineWriter.write(instantsToArchive, Option.of(action ->
deleteAnyLeftOverMarkers(context, action)), Option.of(exceptionHandler));
LOG.info("Deleting archived instants " + instantsToArchive);
- success = deleteArchivedInstants(instantsToArchive, context);
+ deleteArchivedInstants(instantsToArchive, context);
// triggers compaction and cleaning only after archiving action
this.timelineWriter.compactAndClean(context);
} else {
LOG.info("No Instants to archive");
}
- return success;
+ return instantsToArchive.size();
Review Comment:
We can address it in another PR, we kind of do not have atomicity for
current 2 steps:
- flush of archived timeline
- deletion of active metadata files
My roughly thought about it, we can fix the left over active metadata files
(should be deleted from active timeline) in the next round of archiving,
imagine the latest instant time in archived timeline is t10 and the oldest
instant in active timeline is t7, we should retry the deletion of instant
metadat files from t7 ~ 10 at the very beginning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
