[
https://issues.apache.org/jira/browse/FLINK-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated FLINK-15226:
------------------------------
Description:
Trying to move a job which takes in application parameters running on a session
cluster to a job cluster and it fails with the following error in the task
manager
{noformat}
2019-11-23 01:29:16,498 ERROR
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Could not parse
the command line options.
org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the
command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.loadConfiguration(TaskManagerRunner.java:315)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:284)
Caused by: org.apache.commons.cli.MissingOptionException: Missing required
option: c
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 2 more
usage: TaskManagerRunner -c <configuration directory> [-D <property=value>]
-c,--configDir <configuration directory> Directory which contains the
configuration file
flink-conf.yml.
-D <property=value> use value for given property
Exception in thread "main"
org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the
command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.loadConfiguration(TaskManagerRunner.java:315)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:284)
Caused by: org.apache.commons.cli.MissingOptionException: Missing required
option: c
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 2 more{noformat}
Looking at the code, this may be due to adding the {{configDir}} parameter at
the end in \{{taskManagers.sh}} script
[here|https://github.com/apache/flink/blob/d43a1dd397dbc7c0559a07b83380fed164114241/flink-dist/src/main/flink-bin/bin/taskmanager.sh#L73].
Based on the commandline parser
[logic|https://github.com/apache/flink/blob/6258a4c333ce9dba914621b13eac2f7d91f5cb72/flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/parser/CommandLineParser.java#L50],
parsing will stop once a parameter with out {{-D-}} or {{-configDir/-c}} is
encountered which in this case is true. If this diagnosis is correct, can the
{{configDir}} parameter can be added to the beginning instead of the end in the
args list in the job manager shell script? Thanks for your input in advance.
was:
Trying to move a job which takes in application parameters running on a session
cluster to a job cluster and it fails with the following error in the task
manager
{noformat}
2019-11-23 01:29:16,498 ERROR
org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Could not parse
the command line options.
org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the
command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.loadConfiguration(TaskManagerRunner.java:315)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:284)
Caused by: org.apache.commons.cli.MissingOptionException: Missing required
option: c
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 2 more
usage: TaskManagerRunner -c <configuration directory> [-D <property=value>]
-c,--configDir <configuration directory> Directory which contains the
configuration file
flink-conf.yml.
-D <property=value> use value for given property
Exception in thread "main"
org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the
command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.loadConfiguration(TaskManagerRunner.java:315)
at
org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:284)
Caused by: org.apache.commons.cli.MissingOptionException: Missing required
option: c
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 2 more{noformat}
Looking at the code, this may be due to adding the {{configDir}} parameter at
the end in t{{askManagers.sh}} script
[here|https://github.com/apache/flink/blob/d43a1dd397dbc7c0559a07b83380fed164114241/flink-dist/src/main/flink-bin/bin/taskmanager.sh#L73].
Based on the commandline parser
[logic|https://github.com/apache/flink/blob/6258a4c333ce9dba914621b13eac2f7d91f5cb72/flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/parser/CommandLineParser.java#L50],
parsing will stop once a parameter with out {{-D}} or {{--configDir/-c}} is
encountered which in this case is true. If this diagnosis is correct, can the
{{configDir}} parameter can be added to the beginning instead of the end in the
args list in the job manager shell script? Thanks for your input in advance.
> Running job with application parameters fails on a job cluster
> --------------------------------------------------------------
>
> Key: FLINK-15226
> URL: https://issues.apache.org/jira/browse/FLINK-15226
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
> Reporter: Biju Nair
> Priority: Minor
>
> Trying to move a job which takes in application parameters running on a
> session cluster to a job cluster and it fails with the following error in the
> task manager
> {noformat}
> 2019-11-23 01:29:16,498 ERROR
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Could not
> parse the command line options.
> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the
> command line arguments.
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.loadConfiguration(TaskManagerRunner.java:315)
> at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:284)
> Caused by: org.apache.commons.cli.MissingOptionException: Missing required
> option: c
> at
> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> ... 2 more
> usage: TaskManagerRunner -c <configuration directory> [-D <property=value>]
> -c,--configDir <configuration directory> Directory which contains the
> configuration file
> flink-conf.yml.
> -D <property=value> use value for given property
> Exception in thread "main"
> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the
> command line arguments.
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.loadConfiguration(TaskManagerRunner.java:315)
> at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:284)
> Caused by: org.apache.commons.cli.MissingOptionException: Missing required
> option: c
> at
> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> ... 2 more{noformat}
> Looking at the code, this may be due to adding the {{configDir}} parameter at
> the end in \{{taskManagers.sh}} script
> [here|https://github.com/apache/flink/blob/d43a1dd397dbc7c0559a07b83380fed164114241/flink-dist/src/main/flink-bin/bin/taskmanager.sh#L73].
> Based on the commandline parser
> [logic|https://github.com/apache/flink/blob/6258a4c333ce9dba914621b13eac2f7d91f5cb72/flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/parser/CommandLineParser.java#L50],
> parsing will stop once a parameter with out {{-D-}} or {{-configDir/-c}} is
> encountered which in this case is true. If this diagnosis is correct, can
> the {{configDir}} parameter can be added to the beginning instead of the end
> in the args list in the job manager shell script? Thanks for your input in
> advance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
