[
https://issues.apache.org/jira/browse/FLINK-20973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kunnni updated FLINK-20973:
---------------------------
Description:
description: When I submitted flink-on-yarn job on Windows (Operating system of
yarn cluster is Linux), I got error message: “Error: Could not find or load
main class org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint”.
cause: Windows has different classPath delimiter with Linux as in Windows is
";", but in Linux is ":". the main cause is that classPath was built on
Windows, but was splitted on Linux.
solution : replace value of File.pathSeparator with
ApplicationConstants.CLASS_PATH_SEPARATOR in class of
org.apache.flink.yarn.YarnClusterDescriptor and org.apache.flink.yarn.Utils.
sourceCode is displayed below：
{code:java}
private ApplicationReport startAppMaster(......) throws Exception {
......
// classpath assembler
StringBuilder classPathBuilder = new StringBuilder();
if (userJarInclusion == YarnConfigOptions.UserJarInclusion.FIRST) {
for (String userClassPath : userClassPaths)
{ classPathBuilder.append(userClassPath).append(File.pathSeparator); }
}
for (String classPath : systemClassPaths)
{ classPathBuilder.append(classPath).append(File.pathSeparator); }
......
// Setup CLASSPATH and environment variables for ApplicationMaster
final Map<String, String> appMasterEnv = new HashMap<>();
// set user specified app master environment variables
appMasterEnv.putAll(
BootstrapTools.getEnvironmentVariables(ResourceManagerOptions.CONTAINERIZED_MASTER_ENV_PREFIX,
configuration));
// set Flink app class path
appMasterEnv.put(YarnConfigKeys.ENV_FLINK_CLASSPATH,
classPathBuilder.toString());
......
}{code}
was:
When I submitted flink-on-yarn job on Windows (Operating system of yarn cluster
is Linux), I got error message: “Error: Could not find or load main class
org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint”.
then I found out that Windows has different classPath delimiter with Linux as
in Windows is ";", but in Linux is ":". the main cause is that classPath was
built on Windows, but was splitted on Linux.
the solution is to replace value of File.pathSeparator with
ApplicationConstants.CLASS_PATH_SEPARATOR in class of
org.apache.flink.yarn.YarnClusterDescriptor and org.apache.flink.yarn.Utils.
sourceCode is displayed below：
{code:java}
private ApplicationReport startAppMaster(......) throws Exception {
......
// classpath assembler
StringBuilder classPathBuilder = new StringBuilder();
if (userJarInclusion == YarnConfigOptions.UserJarInclusion.FIRST) {
for (String userClassPath : userClassPaths)
{ classPathBuilder.append(userClassPath).append(File.pathSeparator); }
}
for (String classPath : systemClassPaths)
{ classPathBuilder.append(classPath).append(File.pathSeparator); }
......
// Setup CLASSPATH and environment variables for ApplicationMaster
final Map<String, String> appMasterEnv = new HashMap<>();
// set user specified app master environment variables
appMasterEnv.putAll(
BootstrapTools.getEnvironmentVariables(ResourceManagerOptions.CONTAINERIZED_MASTER_ENV_PREFIX,
configuration));
// set Flink app class path
appMasterEnv.put(YarnConfigKeys.ENV_FLINK_CLASSPATH,
classPathBuilder.toString());
......
}{code}
> failed to submit flink-on-yarn job on Windows due to error of classPath
> -----------------------------------------------------------------------
>
> Key: FLINK-20973
> URL: https://issues.apache.org/jira/browse/FLINK-20973
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.10.1, 1.11.3
> Reporter: kunnni
> Priority: Minor
>
> description: When I submitted flink-on-yarn job on Windows (Operating system
> of yarn cluster is Linux), I got error message: “Error: Could not find or
> load main class org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint”.
> cause: Windows has different classPath delimiter with Linux as in Windows is
> ";", but in Linux is ":". the main cause is that classPath was built on
> Windows, but was splitted on Linux.
> solution : replace value of File.pathSeparator with
> ApplicationConstants.CLASS_PATH_SEPARATOR in class of
> org.apache.flink.yarn.YarnClusterDescriptor and org.apache.flink.yarn.Utils.
>
> sourceCode is displayed below：
> {code:java}
> private ApplicationReport startAppMaster(......) throws Exception {
> ......
> // classpath assembler
> StringBuilder classPathBuilder = new StringBuilder();
> if (userJarInclusion == YarnConfigOptions.UserJarInclusion.FIRST) {
> for (String userClassPath : userClassPaths)
> { classPathBuilder.append(userClassPath).append(File.pathSeparator); }
> }
> for (String classPath : systemClassPaths)
> { classPathBuilder.append(classPath).append(File.pathSeparator); }
> ......
> // Setup CLASSPATH and environment variables for ApplicationMaster
> final Map<String, String> appMasterEnv = new HashMap<>();
> // set user specified app master environment variables
> appMasterEnv.putAll(
>
> BootstrapTools.getEnvironmentVariables(ResourceManagerOptions.CONTAINERIZED_MASTER_ENV_PREFIX,
> configuration));
> // set Flink app class path
> appMasterEnv.put(YarnConfigKeys.ENV_FLINK_CLASSPATH,
> classPathBuilder.toString());
> ......
> }{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
