[
https://issues.apache.org/jira/browse/FLINK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15883521#comment-15883521
]
ASF GitHub Bot commented on FLINK-5668:
---------------------------------------
GitHub user billliuatuber opened a pull request:
https://github.com/apache/flink/pull/3413
[FLINK-5668] Reduce dependency on HDFS at job startup time
In current implementation, Job manager writes task manager configuration
into a file and upload it to HDFS. This file's used to bootstrap taskmanager.
In this PR, it switches to use system environment instead of HDFS file to
pass the configuration from job manager to task manager, which reduce the
dependency on HDFS.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/billliuatuber/flink FLINK-5668
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3413.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3413
----
commit 547b333203db8e01737a690e06ad5f5663d7faca
Author: Bill Liu <b...@uber.com>
Date: 2017-02-21T03:27:46Z
[FLINK-5668] reduce hdfs dependency at startup time
----
> Reduce dependency on HDFS at job startup time
> ---------------------------------------------
>
> Key: FLINK-5668
> URL: https://issues.apache.org/jira/browse/FLINK-5668
> Project: Flink
> Issue Type: Improvement
> Components: YARN
> Reporter: Bill Liu
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When create a Flink cluster on Yarn, JobManager depends on HDFS to share
> taskmanager-conf.yaml with TaskManager.
> It's better to share the taskmanager-conf.yaml on JobManager Web server
> instead of HDFS, which could reduce the HDFS dependency at job startup.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
