[
https://issues.apache.org/jira/browse/FLINK-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778777#comment-16778777
]
vinoyang commented on FLINK-5931:
---------------------------------
Hi [~StephanEwen] What do you think about whether we can introduce a backup
storage location configuration item? We have also experienced HDFS failures in
the online environment. If Flink allows configuration of a backed up file
system, it will increase the availability of the entire stream computing
cluster. However, this mechanism involves the problem of metadata
synchronization for storage. Synchronization in a synchronous or asynchronous
manner also needs to be considered. cc [~till.rohrmann]
> Make Flink highly available even if defaultFS is unavailable
> ------------------------------------------------------------
>
> Key: FLINK-5931
> URL: https://issues.apache.org/jira/browse/FLINK-5931
> Project: Flink
> Issue Type: New Feature
> Components: Distributed Coordination
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Priority: Major
>
> In order to use Flink in mission-critical environments, Flink must be
> available even if the {{defaultFS}} is unavailable.
> We have deployed HDFS in HA mode in our production environment. In our
> experience we have experienced performance degradations and downtime when the
> HDFS cluster is being expanded or under maintenances. Under this case it is
> desirable to deploy jobs through alternative filesystem (e.g., S3).
> This jira is to track the improvements to Flink to enable Flink to continue
> to operate even {{defaultFS}} is unavailable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
