Gour Saha created YARN-7543:
-------------------------------
Summary: FileNotFoundException when creating a yarn service due to
broken link under hadoop lib directory
Key: YARN-7543
URL: https://issues.apache.org/jira/browse/YARN-7543
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Gour Saha
The hadoop lib dir had a broken link to a ojdb jar which was not really
required for a YARN service creation. The app submission failed with the below
FNFE. Ideally it should be handled and app should be successfully submitted and
let the app fail if it really needed the jar of the broken link -
{code}
[root@ctr-e134-1499953498516-324910-01-000002 ~]# yarn app -launch gour-sleeper
sleeper
WARNING: YARN_LOG_DIR has been replaced by HADOOP_LOG_DIR. Using value of
YARN_LOG_DIR.
WARNING: YARN_LOGFILE has been replaced by HADOOP_LOGFILE. Using value of
YARN_LOGFILE.
WARNING: YARN_PID_DIR has been replaced by HADOOP_PID_DIR. Using value of
YARN_PID_DIR.
WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of YARN_OPTS.
17/11/21 03:21:58 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
17/11/21 03:21:59 INFO client.RMProxy: Connecting to ResourceManager at
ctr-e134-1499953498516-324910-01-000003.example.com/172.27.47.1:8050
17/11/21 03:22:00 WARN shortcircuit.DomainSocketFactory: The short-circuit
local reads feature cannot be used because libhadoop cannot be loaded.
17/11/21 03:22:00 INFO client.RMProxy: Connecting to ResourceManager at
ctr-e134-1499953498516-324910-01-000003.example.com/172.27.47.1:8050
17/11/21 03:22:00 INFO client.ServiceClient: Loading service definition from
local FS:
/usr/hdp/3.0.0.0-493/hadoop-yarn/yarn-service-examples/sleeper/sleeper.json
17/11/21 03:22:01 INFO client.ServiceClient: Persisted service gour-sleeper at
hdfs://ctr-e134-1499953498516-324910-01-000003.example.com:8020/user/hdfs/.yarn/services/gour-sleeper/gour-sleeper.json
17/11/21 03:22:01 INFO conf.Configuration: resource-types.xml not found
17/11/21 03:22:01 WARN client.ServiceClient: AM log4j property file doesn't
exist: /usr/hdp/3.0.0.0-493/hadoop/conf/yarnservice-log4j.properties
17/11/21 03:22:01 INFO client.ServiceClient: Uploading all dependency jars to
HDFS. For faster submission of apps, pre-upload dependency jars to HDFS using
command: yarn app -enableFastLaunch
Exception in thread "main" java.io.FileNotFoundException: File
/usr/hdp/3.0.0.0-493/hadoop/lib/ojdbc6.jar does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:641)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:867)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:631)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:454)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:365)
at
org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:2399)
at
org.apache.hadoop.yarn.service.utils.CoreFileSystem.submitFile(CoreFileSystem.java:434)
at
org.apache.hadoop.yarn.service.utils.ServiceUtils.putAllJars(ServiceUtils.java:409)
at
org.apache.hadoop.yarn.service.provider.ProviderUtils.addAllDependencyJars(ProviderUtils.java:138)
at
org.apache.hadoop.yarn.service.client.ServiceClient.addJarResource(ServiceClient.java:695)
at
org.apache.hadoop.yarn.service.client.ServiceClient.submitApp(ServiceClient.java:553)
at
org.apache.hadoop.yarn.service.client.ServiceClient.actionCreate(ServiceClient.java:212)
at
org.apache.hadoop.yarn.service.client.ServiceClient.actionLaunch(ServiceClient.java:197)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:447)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:111)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
