[
https://issues.apache.org/jira/browse/FLINK-12343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830343#comment-16830343
]
Till Rohrmann commented on FLINK-12343:
---------------------------------------
I think the {{ResourceManager}} also sets up some local resources when it
creates the {{TaskExecutorContext}}. I guess we should set the same replication
factor for these files as well. A part from that, I think this idea should
work. The important bit is to not define a default value for the Flink option
so that we can fall back to the HDFS default.
> Allow set file.replication in Yarn Configuration
> ------------------------------------------------
>
> Key: FLINK-12343
> URL: https://issues.apache.org/jira/browse/FLINK-12343
> Project: Flink
> Issue Type: Improvement
> Components: Command Line Client, Deployment / YARN
> Affects Versions: 1.6.4, 1.7.2, 1.8.0
> Reporter: Zhenqiu Huang
> Assignee: Zhenqiu Huang
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, FlinkYarnSessionCli upload jars into hdfs with default 3
> replications. From our production experience, we find that 3 replications
> will block big job (256 containers) to launch, when the HDFS is slow due to
> big workload for batch pipelines. Thus, we want to make the factor
> customizable from FlinkYarnSessionCli by adding an option.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
