Till Rohrmann created FLINK-2726:
------------------------------------
Summary: Harmonise Yarn configuration methods to use a single
configuration for all components
Key: FLINK-2726
URL: https://issues.apache.org/jira/browse/FLINK-2726
Project: Flink
Issue Type: Improvement
Components: YARN Client
Reporter: Till Rohrmann
Currently, the {{FlinkYarnClient}} uses different configuration objects to
initialize the {{ApplicationClient}} and the {{ApplicationMaster}}.
The {{ApplicationMaster}} is started with a configuration which is read from
disk by specifying its path. The configuration file is uploaded to HDFS to make
it accessible. Additionally, command line properties are encoded in an
environment variable which will also be transferred to the started Yarn
containers.
The {{ApplicationClient}} is started with a {{Configuration}} object which is
given via a setter to the {{FlinkYarnClient}}. The command line options will be
added to this configuration object. The configuration object does not have to
be the same as the configuration object which is read from disk. This can lead
to serious problems when, for example, the {{akka.framesize}} has to be
consistently configured. At the moment, this only works because the
configuration object is created in the {{FlinkYarnSessionCli}} from the same
file which is also uploaded to HDFS.
I propose to only use a single mean for configuring the different Yarn
components. One could, for example, only set the {{Configuration}} object which
is then written to HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
