[
https://issues.apache.org/jira/browse/KYLIN-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075866#comment-17075866
]
刘磊 commented on KYLIN-4102:
---------------------------
Thanks for your reply. My code has no problem running in local mode, it will
report an error after running on yarn cluster, the jar conflict seems to be on
the datanode, did you have any good way to troubleshoot? I try to print
logs,but still not to find the root cause.
> java.lang.VerifyError: class com.google.common.collect.EmptyImmutableList
> overrides final method toArray.([Ljava/lang/Object;)[Ljava/lang/Object;
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-4102
> URL: https://issues.apache.org/jira/browse/KYLIN-4102
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v2.6.2
> Environment: Kylin2.6.2 , hadoop-2.6.0-cdh5.13.1.1,
> hbase-1.2.0-cdh5.10.1, hive-2.3.4, spark-2.3.2
> Reporter: xiangakun
> Priority: Major
>
> Hi,
>
> When I used spark engine to build cube, and met below error in step #7, it
> seems the guava jar with confliction, is it an issue?
>
> 19/07/19 10:13:55 INFO DAGScheduler: ShuffleMapStage 0 (mapToPair at
> SparkCubingByLayer.java:169) failed in 9.119 s due to Job aborted due to
> stage failure: Task 6 in stage 0.0 failed 4 times, most recent failure: Lost
> task 6.3 in stage 0.0 (TID 18, sd-hadoop-datanode-5
> 1-131.idc.vip.com, executor 2): java.lang.VerifyError: class
> com.google.common.collect.EmptyImmutableList overrides final method
> toArray.([Ljava/lang/Object;)[Ljava/lang/Object; at
> java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.kylin.engine.spark.KylinKryoRegistrator.addClassQuitely(KylinKryoRegistrator.java:312)
> at
> org.apache.kylin.engine.spark.KylinKryoRegistrator.registerClasses(KylinKryoRegistrator.java:109)
> at
> org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:136)
> at
> org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:136)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> at
> org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:136)
> at
> org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:324)
> at
> org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:309)
> at
> org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:218)
> at
> org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:305)
> at
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1$$anonfun$apply$2.apply(TorrentBroadcast.scala:235)
> at scala.Option.getOrElse(Option.scala:121)
> at
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:211)
> at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1347)
> at
> org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:207)
> at
> org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)
> at
> org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)
> at
> org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)
> at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:86)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
> at org.apache.spark.scheduler.Task.run(Task.scala:109)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
