[
https://issues.apache.org/jira/browse/BEAM-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453033#comment-16453033
]
Ben Sidhom commented on BEAM-4063:
----------------------------------
Note that the DistributedCache already supports registering artifacts by
arbitrary names–the registered artifact name mapping is stored in memory in a
map and written out using Java serialization.
> Flink runner supports cluster-wide artifact deployments through the
> Distributed Cache
> -------------------------------------------------------------------------------------
>
> Key: BEAM-4063
> URL: https://issues.apache.org/jira/browse/BEAM-4063
> Project: Beam
> Issue Type: New Feature
> Components: runner-flink
> Reporter: Ben Sidhom
> Priority: Minor
>
> As of now, Flink effectively has a dependency on an external storage system
> for artifact management. This is because the Flink Distributed Cache does not
> actually distribute and cache blobs itself, but rather expects that each node
> in a running cluster has access to a well-known artifact resource.
> We should get this for free whenever
> [https://github.com/apache/flink/pull/5580] is merged (likely in 1.5). For
> now, we will have to defer to external storage systems like GCS or HDFS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
