[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[ https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135711#comment-17135711 ] Yang Wang commented on FLINK-17891: --- I have verified the PR could solve this issue. > FlinkYarnSessionCli sets wrong execution.target type > - > > Key: FLINK-17891 > URL: https://issues.apache.org/jira/browse/FLINK-17891 > Project: Flink > Issue Type: Bug > Components: Command Line Client, Deployment / YARN >Affects Versions: 1.11.0 >Reporter: Shangwen Tang >Assignee: Kostas Kloudas >Priority: Major > Labels: pull-request-available > Attachments: image-2020-05-23-00-59-32-702.png, > image-2020-05-23-01-00-19-549.png > > > I submitted a flink session job at the local YARN cluster, and I found that > the *execution.target* is of the wrong type, which should be of yarn-session > type > !image-2020-05-23-00-59-32-702.png|width=545,height=75! > !image-2020-05-23-01-00-19-549.png|width=544,height=94! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[ https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134301#comment-17134301 ] Kostas Kloudas commented on FLINK-17891: [~tangshangwen] could you please verify that the PR solves the issue you mention? > FlinkYarnSessionCli sets wrong execution.target type > - > > Key: FLINK-17891 > URL: https://issues.apache.org/jira/browse/FLINK-17891 > Project: Flink > Issue Type: Bug > Components: Command Line Client, Deployment / YARN >Affects Versions: 1.11.0 >Reporter: Shangwen Tang >Assignee: Kostas Kloudas >Priority: Major > Labels: pull-request-available > Attachments: image-2020-05-23-00-59-32-702.png, > image-2020-05-23-01-00-19-549.png > > > I submitted a flink session job at the local YARN cluster, and I found that > the *execution.target* is of the wrong type, which should be of yarn-session > type > !image-2020-05-23-00-59-32-702.png|width=545,height=75! > !image-2020-05-23-01-00-19-549.png|width=544,height=94! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[ https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115951#comment-17115951 ] Kostas Kloudas commented on FLINK-17891: Hi [~tangshangwen], as [~fly_in_gis] said, could you describe what exactly is your scenario? I am asking because the fix does not seem to fixing something. You are just removing an option that has to be set to something for correct execution. > FlinkYarnSessionCli sets wrong execution.target type > - > > Key: FLINK-17891 > URL: https://issues.apache.org/jira/browse/FLINK-17891 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN >Affects Versions: 1.11.0 >Reporter: Shangwen Tang >Priority: Major > Attachments: image-2020-05-23-00-59-32-702.png, > image-2020-05-23-01-00-19-549.png > > > I submitted a flink session job at the local YARN cluster, and I found that > the *execution.target* is of the wrong type, which should be of yarn-session > type > !image-2020-05-23-00-59-32-702.png|width=545,height=75! > !image-2020-05-23-01-00-19-549.png|width=544,height=94! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[
https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115624#comment-17115624
]
Yang Wang commented on FLINK-17891:
---
[~tangshangwen] Thanks a lot for reporting this issue. Could you share the full
command how do you start a Flink session cluster and then submit a job to the
existing session? In {{FlinkYarnSessionCli}}, only when the applicationId is
not specified, we assume that it is a per-job cluster. I think it is the
expected behavior.
> FlinkYarnSessionCli sets wrong execution.target type
> -
>
> Key: FLINK-17891
> URL: https://issues.apache.org/jira/browse/FLINK-17891
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
>Affects Versions: 1.11.0
>Reporter: Shangwen Tang
>Priority: Major
> Attachments: image-2020-05-23-00-59-32-702.png,
> image-2020-05-23-01-00-19-549.png
>
>
> I submitted a flink session job at the local YARN cluster, and I found that
> the *execution.target* is of the wrong type, which should be of yarn-session
> type
> !image-2020-05-23-00-59-32-702.png|width=545,height=75!
> !image-2020-05-23-01-00-19-549.png|width=544,height=94!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[ https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114622#comment-17114622 ] Shangwen Tang commented on FLINK-17891: --- Hi [~lzljs3620320] [~jark] , Can you assign this task to me ? I'll try to fix it. Thanks~ > FlinkYarnSessionCli sets wrong execution.target type > - > > Key: FLINK-17891 > URL: https://issues.apache.org/jira/browse/FLINK-17891 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN >Affects Versions: 1.11.0 >Reporter: Shangwen Tang >Priority: Major > Attachments: image-2020-05-23-00-59-32-702.png, > image-2020-05-23-01-00-19-549.png > > > I submitted a flink session job at the local YARN cluster, and I found that > the *execution.target* is of the wrong type, which should be of yarn-session > type > !image-2020-05-23-00-59-32-702.png|width=545,height=75! > !image-2020-05-23-01-00-19-549.png|width=544,height=94! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[ https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114251#comment-17114251 ] Shangwen Tang commented on FLINK-17891: --- This is a simple fix, assign this issue to me and I can submit the code if possible > FlinkYarnSessionCli sets wrong execution.target type > - > > Key: FLINK-17891 > URL: https://issues.apache.org/jira/browse/FLINK-17891 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN >Affects Versions: 1.11.0 >Reporter: Shangwen Tang >Priority: Major > Attachments: image-2020-05-23-00-59-32-702.png, > image-2020-05-23-01-00-19-549.png > > > I submitted a flink session job at the local YARN cluster, and I found that > the *execution.target* is of the wrong type, which should be of yarn-session > type > !image-2020-05-23-00-59-32-702.png|width=545,height=75! > !image-2020-05-23-01-00-19-549.png|width=544,height=94! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17891) FlinkYarnSessionCli sets wrong execution.target type
[
https://issues.apache.org/jira/browse/FLINK-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114246#comment-17114246
]
Shangwen Tang commented on FLINK-17891:
---
My idea is that if we start the flink session job with FlinkYarnSessionCli, we
should set the *execution.target* to be of yarn-session type, not yarn-per-job.
and that's where the problem comes in
`effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnJobClusterExecutor.NAME);`
{code:java}
// FlinkYarnSessionCli.java
@Override
public Configuration applyCommandLineOptionsToConfiguration(CommandLine
commandLine) throws FlinkException {
// we ignore the addressOption because it can only contain "yarn-cluster"
final Configuration effectiveConfiguration = new
Configuration(configuration);
applyDescriptorOptionToConfig(commandLine, effectiveConfiguration);
final ApplicationId applicationId = getApplicationId(commandLine);
if (applicationId != null) {
final String zooKeeperNamespace;
if (commandLine.hasOption(zookeeperNamespace.getOpt())){
zooKeeperNamespace =
commandLine.getOptionValue(zookeeperNamespace.getOpt());
} else {
zooKeeperNamespace = effectiveConfiguration.getString(HA_CLUSTER_ID,
applicationId.toString());
}
effectiveConfiguration.setString(HA_CLUSTER_ID, zooKeeperNamespace);
effectiveConfiguration.setString(YarnConfigOptions.APPLICATION_ID,
ConverterUtils.toString(applicationId));
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnSessionClusterExecutor.NAME);
} else {
effectiveConfiguration.setString(DeploymentOptions.TARGET,
YarnJobClusterExecutor.NAME);
}
...
}{code}
> FlinkYarnSessionCli sets wrong execution.target type
> -
>
> Key: FLINK-17891
> URL: https://issues.apache.org/jira/browse/FLINK-17891
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
>Affects Versions: 1.11.0
>Reporter: Shangwen Tang
>Priority: Major
> Attachments: image-2020-05-23-00-59-32-702.png,
> image-2020-05-23-01-00-19-549.png
>
>
> I submitted a flink session job at the local YARN cluster, and I found that
> the *execution.target* is of the wrong type, which should be of yarn-session
> type
> !image-2020-05-23-00-59-32-702.png|width=545,height=75!
> !image-2020-05-23-01-00-19-549.png|width=544,height=94!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
