[
https://issues.apache.org/jira/browse/FLINK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15901284#comment-15901284
]
Stephan Ewen commented on FLINK-5668:
-------------------------------------
Some background from the Flink side:
- There is quite a bit of work on a new YARN model (as part of FLIP-6) that
deploys TaskManagers differently than the current one. Anything not guarded
through tests has a fair chance of getting lost on the way of eventually
transitioning to that model.
- The high-availability paths will rely on some form of persistent file
system to store generated artifacts (jars, task deployment information, etc).
Given that, I am not sure we gain much my actually not writing the config any
more.
Can we address these issues separately?
1. Have a {{yarn.deploy.fs}} that Flink will use to store new artifacts
2. Use the {{FileSystem}} of the {{Path}} for all other artifacts that are
added, so that it downloads them from other locations (non HDFS) as well
> passing taskmanager configuration through taskManagerEnv instead of file
> ------------------------------------------------------------------------
>
> Key: FLINK-5668
> URL: https://issues.apache.org/jira/browse/FLINK-5668
> Project: Flink
> Issue Type: Improvement
> Components: YARN
> Reporter: Bill Liu
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When create a Flink cluster on Yarn, JobManager depends on HDFS to share
> taskmanager-conf.yaml with TaskManager.
> It's better to share the taskmanager-conf.yaml on JobManager Web server
> instead of HDFS, which could reduce the HDFS dependency at job startup.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
