[
https://issues.apache.org/jira/browse/SPARK-28263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264990#comment-17264990
]
Jack LIN commented on SPARK-28263:
----------------------------------
Just guessing, this probably relates to the settings of `sbt`
> Spark-submit can not find class (ClassNotFoundException)
> --------------------------------------------------------
>
> Key: SPARK-28263
> URL: https://issues.apache.org/jira/browse/SPARK-28263
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell, Spark Submit
> Affects Versions: 2.4.3
> Reporter: Zhiyuan
> Priority: Major
> Labels: beginner
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> I try to run the Main class in my folder using the following code in the
> script:
>
> {code:java}
> spark-shell --class com.navercorp.Main /target/node2vec-0.0.1-SNAPSHOT.jar
> --cmd node2vec ../graph/karate.edgelist --output ../walk/walk.txt {code}
> But it raises the error:
> {code:java}
> 19/07/05 14:39:20 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 19/07/05 14:39:20 WARN deploy.SparkSubmit$$anon$2: Failed to load
> com.navercorp.Main.
> java.lang.ClassNotFoundException: com.navercorp.Main
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:810)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
> at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala){code}
> I have jar file in my folder, this is the structure:
> {code:java}
> 1----node2vec
> 2----node2vec_spark
> 3----main
> 4----resources
> 4----com
> 5----novercorp
> 6----lib
> 7----Main
> 7----Node2vec
> 7----Word2vec
> 2----target
> 3----lib
> 3----classes
> 3----maven-archiver
> 3----node2vec-0.0.1-SNAPSHOT.jar
> 2----graph
> 3---karate.edgelist
> 2----walk
> 3----walk.txt
> {code}
> Also, I attach the structure of jar file:
> {code:java}
> ```META-INF/
> META-INF/MANIFEST.MF
> log4j2.properties
> com/
> com/navercorp/
> com/navercorp/Node2vec$.class
> com/navercorp/Main$Params$$typecreator1$1.class
> com/navercorp/Main$$anon$1$$anonfun$11.class
> com/navercorp/Word2vec$.class
> com/navercorp/Main$$anon$1$$anonfun$8.class
> com/navercorp/Node2vec$$anonfun$randomWalk$1$$anonfun$8.class
> com/navercorp/Node2vec$$anonfun$indexingGraph$4.class
> com/navercorp/Node2vec$$anonfun$initTransitionProb$1.class
> com/navercorp/Main$.class
> com/navercorp/Node2vec$$anonfun$loadNode2Id$1.class
> com/navercorp/Node2vec$$anonfun$14.class
> com/navercorp/Node2vec$$anonfun$readIndexedGraph$2$$anonfun$1.class
> ```{code}
> Could someone give me the advice on how to connect Main class?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
