Hi, everyone,
An annoying problem occurs to me.
When submitting a spark job, the jar file not found exception is
thrown as follows:
does not existread "main" java.io.FileNotFoundException: File
file:/home/algo/shupeng/eeop_bridger/EeopBridger-1.0-SNAPSHOT.jar
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:409)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
at
org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:340)
at
org.apache.spark.deploy.yarn.Client.org$apache$spark$deploy$yarn$Client$$distribute$1(Client.scala:433)
at
org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10.apply(Client.scala:530)
at
org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10.apply(Client.scala:529)
I'm pretty sure the file is located just where it should be.
Here is the submit shell cmd:
/usr/local/hadoop/spark-release/bin/spark-submit --master yarn --deploy-mode
cluster --queue root.algo --name eeop_bridiger --class Bridger
EeopBridger-1.0-SNAPSHOT.jar
Somebody give me some help, please. Many Thanks.
???(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????,?????????????????????
This email message (including any attachments) is confidential and may be
legally privileged. If you have received it by mistake, please notify the
sender by return email and delete this message from your system. Any
unauthorized use or dissemination of this message in whole or in part is
strictly prohibited. Envision Energy Limited and all its subsidiaries shall not
be liable for the improper or incomplete transmission of the information
contained in this email nor for any delay in its receipt or damage to your
system. Envision Energy Limited does not guarantee the integrity of this email
message, nor that this email message is free of viruses, interceptions, or
interference.
