[ https://issues.apache.org/jira/browse/KYLIN-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908627#comment-16908627 ]
shaozhipeng commented on KYLIN-4138: ------------------------------------ cp spark/jars/spark-core_2.11-2.3.2.jar tomcat/lib/ cp spark/jars/scala-library-2.11.8.jar tomcat/lib/ in bin/kylin.sh add: export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$hive_dependency:$HBASE_CLASSPATH_PREFIX download spark source code and build out spark-<version>-yarn-shuffle.jar transfer to $HADOOP_HOME/share/hadoop/yarn/lib update yarn-site.xml add spark_shuffle and YarnShuffleService > kylin2.6.3使用Spark构建Cube,找不到Spark的Class > -------------------------------------- > > Key: KYLIN-4138 > URL: https://issues.apache.org/jira/browse/KYLIN-4138 > Project: Kylin > Issue Type: Bug > Affects Versions: v2.6.3 > Reporter: shaozhipeng > Priority: Major > Attachments: kylin-spark0.png, kylin-spark1.png > > > apache-kylin-2.6.3-bin-hbase1x.tar.gz > $KYLIN_HOME/bin/download-spark.sh > kylin.engine.spark-conf.spark.yarn.archive=hdfs://master:9000/kylin/spark/spark-libs.jar > > 报错: > Caused by: java.lang.NoClassDefFoundError: > org/apache/spark/api/java/function/Function > at > org.apache.kylin.engine.spark.SparkBatchCubingJobBuilder2.<init>(SparkBatchCubingJobBuilder2.java:53) > ~[kylin-engine-spark-2.6.3.jar:2.6.3] > at > org.apache.kylin.engine.spark.SparkBatchCubingEngine2.createBatchCubingJob(SparkBatchCubingEngine2.java:44) > ~[kylin-engine-spark-2.6.3.jar:2.6.3] > at > org.apache.kylin.engine.EngineFactory.createBatchCubingJob(EngineFactory.java:60) > ~[kylin-core-job-2.6.3.jar:2.6.3] > at > org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:234) > ~[kylin-server-base-2.6.3.jar:2.6.3] > at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:202) > ~[kylin-server-base-2.6.3.jar:2.6.3] > at > org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:395) > ~[kylin-server-base-2.6.3.jar:2.6.3] > ... 77 more > Caused by: java.lang.ClassNotFoundException: > org.apache.spark.api.java.function.Function > at > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1928) > ~[catalina.jar:7.0.91] > at > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1771) > ~[catalina.jar:7.0.91] > at > org.apache.kylin.engine.spark.SparkBatchCubingJobBuilder2.<init>(SparkBatchCubingJobBuilder2.java:53) > ~[kylin-engine-spark-2.6.3.jar:2.6.3] > at > org.apache.kylin.engine.spark.SparkBatchCubingEngine2.createBatchCubingJob(SparkBatchCubingEngine2.java:44) > ~[kylin-engine-spark-2.6.3.jar:2.6.3] > at > org.apache.kylin.engine.EngineFactory.createBatchCubingJob(EngineFactory.java:60) > ~[kylin-core-job-2.6.3.jar:2.6.3] > at > org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:234) > ~[kylin-server-base-2.6.3.jar:2.6.3] > at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:202) > ~[kylin-server-base-2.6.3.jar:2.6.3] > at > org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:395) > ~[kylin-server-base-2.6.3.jar:2.6.3] -- This message was sent by Atlassian JIRA (v7.6.14#76016)