[
https://issues.apache.org/jira/browse/KYLIN-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280937#comment-17280937
]
stayblank commented on KYLIN-4904:
----------------------------------
1. my spark-version
!image-2021-02-08-18-36-21-136.png!
2. my hadoop version
!image-2021-02-08-18-37-03-126.png!
3.my hive version
!image-2021-02-08-18-38-10-276.png!
4. kylin.sh start is show and the spark_home is night
!image-2021-02-08-18-41-11-708.png!
5. when i build cubes is error
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.kylin.engine.spark.job.NSparkExecutable.runLocalMode(NSparkExecutable.java:389)
at
org.apache.kylin.engine.spark.job.NSparkExecutable.doWork(NSparkExecutable.java:153)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:94)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error execute
org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:92)
at
org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob.main(ResourceDetectBeforeCubingJob.java:100)
... 13 more
Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'yarn'
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2784)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:493)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:283)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:89)
... 14 more
6. must be the spark_version and hadoop_version and hive_version
matching
> build cubes error
> -------------------
>
> Key: KYLIN-4904
> URL: https://issues.apache.org/jira/browse/KYLIN-4904
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v4.0.0-alpha
> Environment: ubuntu20
> Reporter: stayblank
> Priority: Major
> Attachments: image-2021-02-08-18-36-21-136.png,
> image-2021-02-08-18-37-03-126.png, image-2021-02-08-18-38-10-276.png,
> image-2021-02-08-18-41-11-708.png
>
>
> 1.不添加 `spark-yarn_2.11-2.4.8-SNAPSHOT.jar` 到 $kylin_home/lib,报这个错
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.kylin.engine.spark.job.NSparkExecutable.runLocalMode(NSparkExecutable.java:389)
> at
> org.apache.kylin.engine.spark.job.NSparkExecutable.doWork(NSparkExecutable.java:153)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:94)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: Error execute
> org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:92)
> at
> org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob.main(ResourceDetectBeforeCubingJob.java:100)
> ... 13 more
> Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'yarn'
> at
> org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2784)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:493)
> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
> at scala.Option.getOrElse(Option.scala:121)
> at
> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:283)
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:89)
> ... 14 more
>
>
>
> 2.添加`spark-yarn_2.11-2.4.8-SNAPSHOT.jar` 到 $kylin_home/lib后报这个错
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.kylin.engine.spark.job.NSparkExecutable.runLocalMode(NSparkExecutable.java:389)
> at
> org.apache.kylin.engine.spark.job.NSparkExecutable.doWork(NSparkExecutable.java:153)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:94)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:205)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ExceptionInInitializerError
> at
> org.apache.spark.scheduler.EventLoggingListener$.initEventLog(EventLoggingListener.scala:307)
> at
> org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:126)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:523)
> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
> at scala.Option.getOrElse(Option.scala:121)
> at
> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:283)
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:89)
> at
> org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob.main(ResourceDetectBeforeCubingJob.java:100)
> ... 13 more
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Incompatible
> Jackson version: 2.6.7-3
> at
> com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:67)
> at
> com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
> at
> com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:718)
> at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
> at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala)
> ... 24 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
