[ https://issues.apache.org/jira/browse/SPARK-15847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yesha Vora updated SPARK-15847: ------------------------------- Affects Version/s: 2.0.0 > DecisionTreeRunner example stucks with "NoClassDefFoundError: > org/apache/avro/generic/GenericRecord" > ---------------------------------------------------------------------------------------------------- > > Key: SPARK-15847 > URL: https://issues.apache.org/jira/browse/SPARK-15847 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.0.0 > Reporter: Yesha Vora > > In Spark-2, DecisionTreeRunner example hangs with "NoClassDefFoundError: > org/apache/avro/generic/GenericRecord". > The same application passes in yarn-cluster mode. I'm hitting this issue with > yarn-client mode only. > {code} > spark-submit --class org.apache.spark.examples.mllib.DecisionTreeRunner > --master yarn-client --jars hadoop-lzo-*.jar /xxx/lib/spark-examples_*jar > /tmp/sparkMLLInput/sample_libsvm_data.txt{code} > {code} > 16/05/27 02:37:50 INFO SparkContext: Starting job: countByValue at > DecisionTreeRunner.scala:185 > Exception in thread "dag-scheduler-event-loop" > java.lang.NoClassDefFoundError: org/apache/avro/generic/GenericRecord > at > org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:112) > at > org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:274) > at > org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:259) > at > org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:175) > at > org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects$lzycompute(KryoSerializer.scala:182) > at > org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects(KryoSerializer.scala:178) > at > org.apache.spark.shuffle.sort.SortShuffleManager$.canUseSerializedShuffle(SortShuffleManager.scala:187) > at > org.apache.spark.shuffle.sort.SortShuffleManager.registerShuffle(SortShuffleManager.scala:99) > at org.apache.spark.ShuffleDependency.<init>(Dependency.scala:90) > at org.apache.spark.rdd.ShuffledRDD.getDependencies(ShuffledRDD.scala:91) > at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:234) > at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:232) > at scala.Option.getOrElse(Option.scala:121) > at org.apache.spark.rdd.RDD.dependencies(RDD.scala:232) > at org.apache.spark.scheduler.DAGScheduler.visit$1(DAGScheduler.scala:391) > at > org.apache.spark.scheduler.DAGScheduler.getParentStages(DAGScheduler.scala:403) > at > org.apache.spark.scheduler.DAGScheduler.getParentStagesAndId(DAGScheduler.scala:304) > at > org.apache.spark.scheduler.DAGScheduler.newResultStage(DAGScheduler.scala:339) > at > org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:849) > at > org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1626) > at > org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618) > at > org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607) > at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) > Caused by: java.lang.ClassNotFoundException: > org.apache.avro.generic.GenericRecord > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > .. 23 more{code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org