[
https://issues.apache.org/jira/browse/FLINK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885456#comment-15885456
]
Robert Metzger commented on FLINK-5668:
---------------------------------------
Sorry that I did not look at this JIRA earlier.
[~bill.liu8904] and [~wheat9] if I understand you correctly, you want Flink on
YARN not to use Hadoop's {{fs.defaultFS}} configuration for choosing the
filesystem used to distribute jars and configuration files during deployment?
This basically means that we need to provide a custom configuration key in
Flink (something like {{yarn.deploy.fs}}) to put the stuff to. This would allow
you so use s3 for deploying Flink and hdfs for rocksdb or other state backups.
I'm not able to completely understand FLINK-5631: How can I register an
additional jar from a different file system as a required resource?
> Reduce dependency on HDFS at job startup time
> ---------------------------------------------
>
> Key: FLINK-5668
> URL: https://issues.apache.org/jira/browse/FLINK-5668
> Project: Flink
> Issue Type: Improvement
> Components: YARN
> Reporter: Bill Liu
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When create a Flink cluster on Yarn, JobManager depends on HDFS to share
> taskmanager-conf.yaml with TaskManager.
> It's better to share the taskmanager-conf.yaml on JobManager Web server
> instead of HDFS, which could reduce the HDFS dependency at job startup.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
