[
https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114246#comment-17114246
]
Shangwen Tang edited comment on FLINK-17891 at 5/22/20, 5:14 PM:
-----------------------------------------------------------------
My idea is that if we start the flink session job with FlinkYarnSessionCli, we
should set the *execution.target* to be of yarn-session type, not yarn-per-job.
and the problem is in this line
{code:java}
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnJobClusterExecutor.NAME);{code}
{code:java}
// FlinkYarnSessionCli.java
@Override
public Configuration applyCommandLineOptionsToConfiguration(CommandLine
commandLine) throws FlinkException {
// we ignore the addressOption because it can only contain "yarn-cluster"
final Configuration effectiveConfiguration = new
Configuration(configuration);
applyDescriptorOptionToConfig(commandLine, effectiveConfiguration);
final ApplicationId applicationId = getApplicationId(commandLine);
if (applicationId != null) {
final String zooKeeperNamespace;
if (commandLine.hasOption(zookeeperNamespace.getOpt())){
zooKeeperNamespace =
commandLine.getOptionValue(zookeeperNamespace.getOpt());
} else {
zooKeeperNamespace = effectiveConfiguration.getString(HA_CLUSTER_ID,
applicationId.toString());
}
effectiveConfiguration.setString(HA_CLUSTER_ID, zooKeeperNamespace);
effectiveConfiguration.setString(YarnConfigOptions.APPLICATION_ID,
ConverterUtils.toString(applicationId));
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnSessionClusterExecutor.NAME);
} else {
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnJobClusterExecutor.NAME);
}
...
}{code}
was (Author: tangshangwen):
My idea is that if we start the flink session job with FlinkYarnSessionCli, we
should set the *execution.target* to be of yarn-session type, not yarn-per-job.
and the problem is in this line
{{}}{{{noformat}
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnJobClusterExecutor.NAME);}}{{{noformat}}}
{code:java}
// FlinkYarnSessionCli.java
@Override
public Configuration applyCommandLineOptionsToConfiguration(CommandLine
commandLine) throws FlinkException {
// we ignore the addressOption because it can only contain "yarn-cluster"
final Configuration effectiveConfiguration = new
Configuration(configuration);
applyDescriptorOptionToConfig(commandLine, effectiveConfiguration);
final ApplicationId applicationId = getApplicationId(commandLine);
if (applicationId != null) {
final String zooKeeperNamespace;
if (commandLine.hasOption(zookeeperNamespace.getOpt())){
zooKeeperNamespace =
commandLine.getOptionValue(zookeeperNamespace.getOpt());
} else {
zooKeeperNamespace = effectiveConfiguration.getString(HA_CLUSTER_ID,
applicationId.toString());
}
effectiveConfiguration.setString(HA_CLUSTER_ID, zooKeeperNamespace);
effectiveConfiguration.setString(YarnConfigOptions.APPLICATION_ID,
ConverterUtils.toString(applicationId));
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnSessionClusterExecutor.NAME);
} else {
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnJobClusterExecutor.NAME);
}
...
}{code}
> FlinkYarnSessionCli sets wrong execution.target type
> -----------------------------------------------------
>
> Key: FLINK-17891
> URL: https://issues.apache.org/jira/browse/FLINK-17891
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.11.0
> Reporter: Shangwen Tang
> Priority: Major
> Attachments: image-2020-05-23-00-59-32-702.png,
> image-2020-05-23-01-00-19-549.png
>
>
> I submitted a flink session job at the local YARN cluster, and I found that
> the *execution.target* is of the wrong type, which should be of yarn-session
> type
> !image-2020-05-23-00-59-32-702.png|width=545,height=75!
> !image-2020-05-23-01-00-19-549.png|width=544,height=94!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
