[
https://issues.apache.org/jira/browse/FLINK-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17156588#comment-17156588
]
Yumeng Zhang commented on FLINK-18551:
--------------------------------------
Hi Xintong, your understanding is correct. In my case, I run my job on Yarn in
perjob mode. I find sometimes it takes 2-3s for Yarn nodemanager to download
the flink-conf.yaml file from hdfs before launching a container. I guess the
reason perhaps is my hdfs cluster's load is high, so I thought we may avoid
downloading the flink-conf.yaml file on Yarn mode to save 2-3s launching a
container in some bad cases. Yes, I agree it indeed introduces some complexity
and maintenance overhead.
> Don't need to download flink-conf.yaml when launching taskmanager on Yarn mode
> ------------------------------------------------------------------------------
>
> Key: FLINK-18551
> URL: https://issues.apache.org/jira/browse/FLINK-18551
> Project: Flink
> Issue Type: Improvement
> Affects Versions: 1.10.1, 1.11.0
> Reporter: Yumeng Zhang
> Priority: Minor
>
> In current flink version, launching a taskmanager on Yarn will follow these
> steps:
> 1. download flink-conf.yaml file to the local working directory
> 2. get Configuration object by reading local flink-conf.yaml file
> We could reduce the IO operation by setting flinkConfig to container env when
> createTaskExecutorContext in jobmanager side and getting Configuration object
> by system env in taskmanager side. This way we don't need to download
> flink-conf.yaml when launching taskmanager on Yarn and could accelerate the
> start process a little bit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
