我架设了 KYLIN 最新版本的,sample 测试是可以正常使用spark。
我自己也创建了cube 也跑成功了一个,但是后面的建立的cube 就失败了,我从spark 的history 查看日志,错误如下
java.lang.IllegalAccessError: tried to access class
org.apache.kylin.dimension.BooleanDimEnc$BooleanSerializer from class
org.apache.kylin.engine.spark.KylinKryoRegistrator
java.lang.IllegalAccessError: tried to access class
org.apache.kylin.dimension.BooleanDimEnc$BooleanSerializer from class
org.apache.kylin.engine.spark.KylinKryoRegistrator
at
org.apache.kylin.engine.spark.KylinKryoRegistrator.kylinClassByReflection1(KylinKryoRegistrator.java:173)
at
org.apache.kylin.engine.spark.KylinKryoRegistrator.registerClasses(KylinKryoRegistrator.java:85)
at
org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:134)
at
org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:134)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at
org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:134)
at
org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:289)
at
org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:274)
at
org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:184)
at
org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:287)
at
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:225)
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1269)
at
org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206)
at
org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)
at
org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)
at
org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:86)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
请问一下这个错误是 kylin 的配置问题呢 还是 spark 的问题,现在很迷惑。
而且我用spark 执行成功的 cube ,spark 只能用 yarn+client ,用 yarn+clucter 失败
请指教,谢谢!!
--
Sent from: http://apache-kylin.74782.x6.nabble.com/