[
https://issues.apache.org/jira/browse/YARN-9190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741752#comment-16741752
]
Zhankun Tang commented on YARN-9190:
------------------------------------
[~billie.rinaldi] , Thanks for the reply!
One thing I forget to mention is that I use the yarn script to run submarine
job for both 3.2.0RC1 and trunk(3.3) build.
{code:java}
yarn jar
$HADOOP_COMMON_HOME/share/hadoop/yarn/hadoop-yarn-submarine-${VERSION}.jar job
run ...{code}
It's not clear to me that given both branches' "yarn" script already set the
"service.libdir", why the above same submarine run script fails in 3.2 RC1?
> [Submarine] Submarine job will fail to run as a first job on a new created
> Hadoop 3.2.0 RC1 cluster
> ---------------------------------------------------------------------------------------------------
>
> Key: YARN-9190
> URL: https://issues.apache.org/jira/browse/YARN-9190
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Zhankun Tang
> Assignee: Sunil Govindan
> Priority: Minor
>
> This issue was found when verifying submarine in Hadoop 3.2.0 RC1 planning.
> The reproduce steps are:
> # Init a new HDFS and YARN (LinuxContainerExecutor and Docker enabled)
> # Before run any other yarn service job, use yarn user to submit a submarine
> job
> The job will fail with below error:
>
> {code:java}
> LogType:serviceam-err.txt
> LogLastModifiedTime:Thu Jan 10 21:15:23 +0800 2019
> LogLength:86
> LogContents:
> Error: Could not find or load main class
> org.apache.hadoop.yarn.service.ServiceMaster
> End of LogType:serviceam-err.txt
> {code}
> This seems because the dependencies are not ready as the service client
> reported:
> {code:java}
> 2019-01-10 21:50:47,380 WARN client.ServiceClient: Property
> yarn.service.framework.path has a value
> /yarn-services/3.2.0/service-dep.tar.gz, but is not a valid file
> 2019-01-10 21:50:47,381 INFO client.ServiceClient: Uploading all dependency
> jars to HDFS. For faster submission of apps, set config property
> yarn.service.framework.path to the dependency tarball location. Dependency
> tarball can be uploaded to any HDFS path directly or by using command: yarn
> app -enableFastLaunch [<Destination Folder>]{code}
>
> When this error happens, I found that there is no “/yarn-services” directory
> created in HDFS.
> But after I run “yarn app -launch my-sleeper sleeper”, the “/yarn-services”
> created in HDFS and then the submarine job can run successfully.
> {code:java}
> yarn@master0-VirtualBox:~/apache-hadoop-install-dir/hadoop-dev-workspace$
> hdfs dfs -ls /yarn-services/3.2.0/*
> -rwxr-xr-x 1 yarn supergroup 93596476 2019-01-11 08:23
> /yarn-services/3.2.0/service-dep.tar.gz{code}
> It seems an issue of yarn service in 3.2.0 RC1 and I files this Jira to track
> it.
>
> And verified that trunk branch doesn't have this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
