[
https://issues.apache.org/jira/browse/FLINK-7057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072335#comment-16072335
]
ASF GitHub Bot commented on FLINK-7057:
---------------------------------------
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/4238#discussion_r125219368
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobCache.java ---
@@ -255,8 +403,39 @@ public int getPort() {
return serverAddress.getPort();
}
+ /**
+ * Cleans up BLOBs which are not referenced anymore.
+ */
+ @Override
+ public void run() {
+ synchronized (lockObject) {
+ Iterator<Map.Entry<JobID, RefCount>> entryIter =
jobRefCounters.entrySet().iterator();
+
+ while (entryIter.hasNext()) {
+ Map.Entry<JobID, RefCount> entry =
entryIter.next();
+ RefCount ref = entry.getValue();
+
+ if (ref.references <= 0 &&
System.currentTimeMillis() >= ref.keepUntil) {
+ JobID jobId = entry.getKey();
+
+ final File localFile =
+ new
File(BlobUtils.getStorageLocationPath(storageDir.getAbsolutePath(), jobId));
+ try {
+
FileUtils.deleteDirectory(localFile);
+ // let's only remove this
directory from cleanup if the cleanup was successful
+ entryIter.remove();
+ } catch (Throwable t) {
+ LOG.warn("Failed to locally
delete job directory {}", localFile.getAbsolutePath());
--- End diff --
Swallowing of exception
> move BLOB ref-counting from LibraryCacheManager to BlobCache
> ------------------------------------------------------------
>
> Key: FLINK-7057
> URL: https://issues.apache.org/jira/browse/FLINK-7057
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination, Network
> Affects Versions: 1.4.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
>
> Currently, the {{LibraryCacheManager}} is doing some ref-counting for JAR
> files managed by it. Instead, we want the {{BlobCache}} to do that itself for
> all job-related BLOBs. Also, we do not want to operate on a per-{{BlobKey}}
> level but rather per job. Therefore, the cleanup process should be adapted,
> too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
