Till Rohrmann created FLINK-25436:
-------------------------------------
Summary: Allow BlobServer/BlobCache to clean up unused blobs after
recovering from working directory
Key: FLINK-25436
URL: https://issues.apache.org/jira/browse/FLINK-25436
Project: Flink
Issue Type: Sub-task
Components: Runtime / Coordination
Affects Versions: 1.15.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Fix For: 1.15.0
In order to let the {{BlobServer}} and the {{BlobCache}} properly clean up
unused blobs that are recovered from the working directory, we have to register
them for clean up and offer hooks to delete irrelevant job artifacts.
I propose to scan the blobStorage directory at startup and to register for
transient blobs the expiry timeouts. Moreover, for the {{BlobServer}} we need
to add a {{retainJobs}} method that deletes all jobs that are not in the given
list of {{JobIDs}}. Last but not least we also need to register the permanent
blobs in the {{PermanentBlobCacheService}} so that they get timed out if not
used anymore.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
