Zhiyuan created SPARK-28263:
-------------------------------
Summary: Spark-submit can not find class (ClassNotFoundException)
Key: SPARK-28263
URL: https://issues.apache.org/jira/browse/SPARK-28263
Project: Spark
Issue Type: Bug
Components: Spark Shell, Spark Submit
Affects Versions: 2.4.3
Reporter: Zhiyuan
I try to run the Main class in my folder using the following code in the script:
{code:java}
spark-shell --class com.navercorp.Main /target/node2vec-0.0.1-SNAPSHOT.jar
--cmd node2vec ../graph/karate.edgelist --output ../walk/walk.txt {code}
But it raises the error:
{code:java}
19/07/05 14:39:20 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
19/07/05 14:39:20 WARN deploy.SparkSubmit$$anon$2: Failed to load
com.navercorp.Main.
java.lang.ClassNotFoundException: com.navercorp.Main
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:810)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala){code}
I have jar file in my folder, this is the structure:
{code:java}
1----node2vec
2----node2vec_spark
3----main
4----resources
4----com
5----novercorp
6----lib
7----Main
7----Node2vec
7----Word2vec
2----target
3----lib
3----classes
3----maven-archiver
3----node2vec-0.0.1-SNAPSHOT.jar
2----graph
3---karate.edgelist
2----walk
3----walk.txt
{code}
Also, I attach the structure of jar file:
{code:java}
```META-INF/
META-INF/MANIFEST.MF
log4j2.properties
com/
com/navercorp/
com/navercorp/Node2vec$.class
com/navercorp/Main$Params$$typecreator1$1.class
com/navercorp/Main$$anon$1$$anonfun$11.class
com/navercorp/Word2vec$.class
com/navercorp/Main$$anon$1$$anonfun$8.class
com/navercorp/Node2vec$$anonfun$randomWalk$1$$anonfun$8.class
com/navercorp/Node2vec$$anonfun$indexingGraph$4.class
com/navercorp/Node2vec$$anonfun$initTransitionProb$1.class
com/navercorp/Main$.class
com/navercorp/Node2vec$$anonfun$loadNode2Id$1.class
com/navercorp/Node2vec$$anonfun$14.class
com/navercorp/Node2vec$$anonfun$readIndexedGraph$2$$anonfun$1.class
```{code}
Could someone give me the advice on how to connect Main class?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
