spuru9 commented on code in PR #29008:
URL: https://github.com/apache/flink/pull/29008#discussion_r4500517369
##########
flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServerArchiveFetcher.java:
##########
@@ -229,23 +328,46 @@ void scanArchives(
continue;
}
+ if (cachedRetainedJobs > 0
+ && fileOrderedIndexOnModifiedTime >
cachedRetainedJobs) {
+ // the remote archive stays, only evict what an
earlier scan cached
+ if
(cachedArchivesPerRefreshDirectory.get(refreshDir).contains(archiveId)) {
+ archivesToEvictLocally
+ .computeIfAbsent(refreshDir, ignored ->
new HashSet<>())
+ .add(archiveId);
+ }
+ continue;
+ }
+
if (fetch) {
- fetchArchive(refreshDir, archiveId, archivePath,
archiveLoadMode, events);
+ synchronized (localStateLock) {
Review Comment:
minor: Every job request now takes localStateLock, but the lock is also held
across I/O: the scan's archive read in fetchArchive, deleting an evicted job's
files, and rebuilding the overview. With big archives in minikube an in-window
job's /config waited for whichever of those was running (a few hundred ms
instead of tens), and the scan part applies even when cached-retained-jobs is
unset, so with S3 or many jobs this would show up for default LAZY users too.
Could the I/O run outside the lock with only the bookkeeping inside it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
