[
https://issues.apache.org/jira/browse/FLINK-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178992#comment-16178992
]
ASF GitHub Bot commented on FLINK-7068:
---------------------------------------
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4358
After an offline discussion with @tillrohrmann, we agreed to have more of
the transient vs. permanent BLOB handling inside the `BlobServer` by including
the type into the `BlobKey`. The caches, however, remain separate (with a
common base class for shared code) because of the different cache guarantees
they give. In addition, we decided upon having a get-and-delete functionality
for transient BLOBs (as planned by a TODO item previously).
This means the following changes:
- the `BlobServer` deletes transient BLOBs after they have been
successfully downloaded to a BLOB cache (the client acknowledges that)
- the `TransientBlobCache` and `PermanentBlobCache` differ in the
functionality they provide:
- `PermanentBlobCache` uses ref-counting for cleaning up its resources
- `TransientBlobCache` imposes the user to manually delete requested BLOB
files (from the cache!)
- `TransientBlobCache` allows uploading (transient) BLOBs
- `TransientBlobCache` allows non-job-related BLOBs
- both have a GET method: `#getTransientFile()` vs. `#getPermanentFile()`
- irrespective of the actual cache being used, the `BlobServer` decides
whether to delete the BLOB (transient BLOBs only) or not (permanent BLOBs)
- theoretically, a permanent BLOB could also be retrieved from the
`TransientBlobCache` (and the other way around if job-related) - this behaviour
is not used though but basic tests exist to cover it
> change BlobService sub-classes for permanent and transient BLOBs
> ----------------------------------------------------------------
>
> Key: FLINK-7068
> URL: https://issues.apache.org/jira/browse/FLINK-7068
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination, Network
> Affects Versions: 1.4.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
>
> A {{PermanentBlobStore}} should resemble use cases for BLOBs that are
> permanently stored for a job's life time (HA and non-HA).
> A {{TransientBlobStore}} should reflect BLOB offloading for logs, RPC, etc.
> which even does not have to be reflected by files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
