设置了kylin.engine.spark-conf.spark.yarn.archive，我也很奇怪，看上去就是没有加载到spark-libs.jar。
## manually upload spark-assembly jar to HDFS and then set this property will avoid repeatedly uploading jar at runtime kylin.engine.spark-conf.spark.yarn.archive=hdfs://master:9000/kylin/spark/spark-libs.jar $ hadoop fs -ls hdfs://master:9000/kylin/spark/ Found 1 items -rw-r--r-- 3 admin supergroup 200878688 2019-08-14 17:16 hdfs://master:9000/kylin/spark/spark-libs.jar 在 2019-08-14 14:46:37，"Chao Long" <chao.long0...@gmail.com> 写道： 这个参数配置了么，kylin.engine.spark-conf.spark.yarn.archive，是否配置为自己集群所在的hdfs目录，而不是文档上的hdfs://sandbox.hortonworks.com:8020 On Wed, Aug 14, 2019 at 2:37 PM 邵志鹏 <bobr...@163.com> wrote: 2019-08-14T14:31:09,188 INFO [http-bio-7070-exec-7] org.apache.kylin.cube.CubeManager - Updating cube instance 'kylin_sales_cube_clone_spark' 2019-08-14T14:31:09,188 INFO [http-bio-7070-exec-7] org.apache.kylin.cube.CubeManager - Remove segment kylin_sales_cube_clone_spark[20120101000000_20140201235500] Action - Delete Segement can Remove segment , and then rebuild it. Main problem is Caused by: java.lang.ClassNotFoundException: org.apache.spark.api.java.function.Function。 不确定是否是jar报冲突，spark下面所有的jar都一起打包放到hdfs了，里面有spark-core。 spark版本：2.3.2 spark-core_2.11-2.3.2 在 2019-08-14 14:08:03，"邵志鹏" <bobr...@163.com> 写道： Hi kylin team: kylin-2.6.3 使用MapReduce已经成功构建smaple cube。 然后clone了一份kylin_sales_cube修改为Spark，build 报错。 Spark是使用脚本下载的，配置参考了http://kylin.apache.org/cn/docs/tutorial/cube_spark.html。 $KYLIN_HOME/bin/download-spark.sh jar cv0f spark-libs.jar -C $KYLIN_HOME/spark/jars/ . hadoop fs -mkdir -p /kylin/spark/ hadoop fs -put spark-libs.jar /kylin/spark/ ... 报错日志： Caused by: java.lang.NoClassDefFoundError: org/apache/spark/api/java/function/Function at org.apache.kylin.engine.spark.SparkBatchCubingJobBuilder2.<init>(SparkBatchCubingJobBuilder2.java:53) ~[kylin-engine-spark-2.6.3.jar:2.6.3] at org.apache.kylin.engine.spark.SparkBatchCubingEngine2.createBatchCubingJob(SparkBatchCubingEngine2.java:44) ~[kylin-engine-spark-2.6.3.jar:2.6.3] at org.apache.kylin.engine.EngineFactory.createBatchCubingJob(EngineFactory.java:60) ~[kylin-core-job-2.6.3.jar:2.6.3] at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:234) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:202) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:395) ~[kylin-server-base-2.6.3.jar:2.6.3] ... 77 more Caused by: java.lang.ClassNotFoundException: org.apache.spark.api.java.function.Function at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1928) ~[catalina.jar:7.0.91] at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1771) ~[catalina.jar:7.0.91] at org.apache.kylin.engine.spark.SparkBatchCubingJobBuilder2.<init>(SparkBatchCubingJobBuilder2.java:53) ~[kylin-engine-spark-2.6.3.jar:2.6.3] at org.apache.kylin.engine.spark.SparkBatchCubingEngine2.createBatchCubingJob(SparkBatchCubingEngine2.java:44) ~[kylin-engine-spark-2.6.3.jar:2.6.3] at org.apache.kylin.engine.EngineFactory.createBatchCubingJob(EngineFactory.java:60) ~[kylin-core-job-2.6.3.jar:2.6.3] at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:234) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:202) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:395) ~[kylin-server-base-2.6.3.jar:2.6.3] 报错之后，再去提交build，又报新的错： Caused by: org.apache.kylin.rest.exception.BadRequestException: The cube kylin_sales_cube_clone_spark has segments [kylin_sales_cube_clone_spark[20120101000000_20140201235500]], but none of them is READY. It's not allowed for parallel building at org.apache.kylin.rest.service.JobService.checkAllowParallelBuilding(JobService.java:422) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:221) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:202) ~[kylin-server-base-2.6.3.jar:2.6.3] at org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:395) ~[kylin-server-base-2.6.3.jar:2.6.3] ... 77 more Monitor里面是没有新的构建任务的。 Thanks. By 邵志鹏
