[
https://issues.apache.org/jira/browse/FLINK-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17156527#comment-17156527
]
Xintong Song commented on FLINK-18551:
--------------------------------------
Hi [~Yumeng],
Thanks for reporting this.
Correct me if I'm wrong. Are you suggesting serialize the {{Configuration}}
object on the JM side, and deserialize it on the TM side, to avoid downloading
the {{flink-conf.yaml}} file?
I'm not sure about this proposal. I wonder how many time we can save from this
proposed change. According to our experiences, downloading the config file
takes almost no time.
On the other hand, passing the configuration object through environment
variables introduces more complexity and maintain overhead, by requiring TM to
load configurations with a different approach, instead of reusing the approach
for other deployments. Such complexity does not seem necessary to me.
Maybe you can share some insight on how this proposed improvement is necessary
to you.
> Don't need to download flink-conf.yaml when launching taskmanager on Yarn mode
> ------------------------------------------------------------------------------
>
> Key: FLINK-18551
> URL: https://issues.apache.org/jira/browse/FLINK-18551
> Project: Flink
> Issue Type: Improvement
> Affects Versions: 1.10.1, 1.11.0
> Reporter: Yumeng Zhang
> Priority: Minor
>
> In current flink version, launching a taskmanager on Yarn will follow these
> steps:
> 1. download flink-conf.yaml file to the local working directory
> 2. get Configuration object by reading local flink-conf.yaml file
> We could reduce the IO operation by setting flinkConfig to container env when
> createTaskExecutorContext in jobmanager side and getting Configuration object
> by system env in taskmanager side. This way we don't need to download
> flink-conf.yaml when launching taskmanager on Yarn and could accelerate the
> start process a little bit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
