[
https://issues.apache.org/jira/browse/KYLIN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581591#comment-17581591
]
Xiaoxiang Yu edited comment on KYLIN-5231 at 8/19/22 1:55 AM:
--------------------------------------------------------------
[~sandeepputrevu] Hi, Kylin 3.X doesn't support Spark 3.X(only Spark 2.4.x or
earlier), and Kylin 4.X doesn't support Spark 3.2.X or higher version(only
Spark 2.4.x or Spark 3.1.x).
Kylin 5.0 alpha, which is now under development, will support Spark 3.2+.
was (Author: xxyu):
[~sandeepputrevu] Hi, Kylin 3.X doesn't support Spark 3.X(only Spark 2.4.x),
and Kylin 4.X doesn't support Spark 3.2.X or higher version(only Spark 2.4.x or
Spark 3.1.x).
> RuntimeException in cube build with Spark engine
> ------------------------------------------------
>
> Key: KYLIN-5231
> URL: https://issues.apache.org/jira/browse/KYLIN-5231
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v3.1.2
> Reporter: Sandeep Putrevu
> Assignee: Xiaoxiang Yu
> Priority: Major
>
> Cube build is failing at *Step #7 : Build Cube with Spark*
>
> Our configuration:
> Spark version : 3.2.1
> Hadoop version : 3.1.1.3.1.4.0
> Kylin version : 3.1.2
> Error :
> Exception in thread "main" java.lang.RuntimeException: error execute
> org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause:
> scala.reflect.ClassTag$$anon$1
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassTag$$anon$1
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:142)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 13 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
