Zhankun Tang created YARN-9190:
----------------------------------
Summary: [Submarine] Submarine job will fail to run as a first job
on a new created Hadoop 3.2.0 RC1
Key: YARN-9190
URL: https://issues.apache.org/jira/browse/YARN-9190
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Zhankun Tang
Assignee: Sunil Govindan
This issue was found when verifying submarine in Hadoop 3.2.0 RC1 planning. The
reproduce steps are:
# Init a new HDFS and YARN (LinuxContainerExecutor and Docker enabled)
# Before run any other yarn service job, use yarn user to submit a submarine
job
The job will fail with below error:
{code:java}
LogType:serviceam-err.txt
LogLastModifiedTime:Thu Jan 10 21:15:23 +0800 2019
LogLength:86
LogContents:
Error: Could not find or load main class
org.apache.hadoop.yarn.service.ServiceMaster
End of LogType:serviceam-err.txt
{code}
This seems because the dependencies are not ready as the service client
reported:
{code:java}
2019-01-10 21:50:47,380 WARN client.ServiceClient: Property
yarn.service.framework.path has a value
/yarn-services/3.2.0/service-dep.tar.gz, but is not a valid file
2019-01-10 21:50:47,381 INFO client.ServiceClient: Uploading all dependency
jars to HDFS. For faster submission of apps, set config property
yarn.service.framework.path to the dependency tarball location. Dependency
tarball can be uploaded to any HDFS path directly or by using command: yarn app
-enableFastLaunch [<Destination Folder>]{code}
When this error happens, I found that there is no “/yarn-services” directory
created in HDFS.
But after I run “yarn app -launch my-sleeper sleeper”, the “/yarn-services”
created in HDFS and then the submarine job can run successfully.
It seems an issue of yarn service in 3.2.0 RC1 and I files this Jira to track
it.
And verified that trunk branch doesn't have this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
