Copilot commented on code in PR #29008:
URL: https://github.com/apache/flink/pull/29008#discussion_r4262739981
##########
flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServerArchiveFetcher.java:
##########
@@ -368,6 +421,27 @@ List<ArchiveEvent>
cleanupArchivesBeyondRetainedLimit(Map<Path, Set<Path>> archi
return cleanupExpiredArchives(allArchiveIdsToRemove);
}
+ /**
+ * Cleans up archives beyond {@link
HistoryServerOptions#HISTORY_SERVER_RETAINED_JOBS} from the
+ * local cache only. Unlike {@link #cleanupArchivesBeyondRetainedLimit},
the remote archive is
+ * left untouched so that it remains fetchable on demand, e.g. via {@link
+ * HistoryServerOptions.HistoryServerArchiveLoadMode#LAZY} mode.
+ */
+ List<ArchiveEvent> cleanupLocalArchivesBeyondRetainedLimit(
+ Map<Path, Set<Path>> archivesToRemove) {
+ Map<Path, Set<String>> allArchiveIdsToRemove = new HashMap<>();
+
+ for (Map.Entry<Path, Set<Path>> pathSetEntry :
archivesToRemove.entrySet()) {
+ HashSet<String> archiveIdsToRemove = new HashSet<>();
+ for (Path archive : pathSetEntry.getValue()) {
+ archiveIdsToRemove.add(archive.getName());
+ }
+ allArchiveIdsToRemove.put(pathSetEntry.getKey(),
archiveIdsToRemove);
+ }
+
+ return cleanupExpiredArchives(allArchiveIdsToRemove);
Review Comment:
This delegates to `cleanupExpiredArchives`, which cancels both common and
individual lazy-fetch futures and deletes their partially written entries. In
LAZY mode the fixed-delay scan can therefore classify a remotely retained,
out-of-window job while a request is fetching it on `individualFetchExecutor`;
the waiting request then observes cancellation/missing content instead of the
promised on-demand archive. Coordinate cleanup with active individual fetches
(for example, defer cleanup for an in-flight job under synchronization and
retry on a later scan) rather than cancelling the request-driven fetch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
