Jiaxing Chen created FLINK-33724:
------------------------------------
Summary: Application mode doesn't support multiple lines argument
Key: FLINK-33724
URL: https://issues.apache.org/jira/browse/FLINK-33724
Project: Flink
Issue Type: Bug
Components: Deployment / YARN
Affects Versions: 1.18.0
Reporter: Jiaxing Chen
When running a job in application mode, program arguments are stored in the
flink-conf.yaml file. These arguments will be loaded when deploying an
application on YARN. However multiple lines argument cannot be loaded properly.
In my case, I submit a job with sql which is a multiple lines argument:
{code:java}
/bin/flink run-application
-t yarn-application
...
myjob.jar
"INSERT INTO tableA
SELECT
a,
b,
c
FROM tableB;"{code}
In flink-conf.yaml，it saved as:
{code:java}
$internal.application.program-args： "INSERT INTO tableA
SELECT
a,
b,
c
FROM tableB;"{code}
And produced some warning logs when loaded from flink-conf.yaml :
{code:java}
INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: $internal.application.program-args, "INSERT INTO tableA
SELECT a,
WARN org.apache.flink.configuration.GlobalConfiguration [] - Error
while trying to split key and value in configuration file /{working
dir}/flink-conf.yaml:{line num} "SELECT "
...{code}
I dig into the source code and find out that the reader and writer methods of
YAML file cannot properly parse YAML format. Maybe it would be better to use
some third-party tools, such as snakeyaml, to do this job.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
