Hi, hubert stefani.
It seems the conflict of the jar, maybe you can try to put the
jersey-client jar in the yarn service into the jars directory of the Spark.
发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用
________________________________
发件人: hubert stefani <[email protected]>
发送时间: Monday, January 28, 2019 10:24:58 PM
收件人: [email protected]
主题: Error when activating spark interactive on v2.6
Hi,
It seems that when we activate the property hereunder , the spark build
job fails. Have you noticed such behaviour ? Should I open an issue on Jira ?
Context :
Kylin : v2.6
AWS : EMR 5.18
Property :
kylin.engine.spark-conf.spark.hadoop.yarn.timeline-service.enabled
Error Log :
2019-01-28 10:50:36,384 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
2019-01-28 10:50:36 WARN SparkConf:66 - The configuration key
'spark.yarn.executor.memoryOverhead' has been deprecated as of Spark 2.3 and
may be removed in the future. Please use the new key
'spark.executor.memoryOverhead' instead.
2019-01-28 10:50:37,732 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
Exception in thread "main" java.lang.NoClassDefFoundError:
com/sun/jersey/api/client/config/ClientConfig
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at
org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:151)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1152)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1520)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
Caused by: java.lang.ClassNotFoundException:
com.sun.jersey.api.client.config.ClientConfig
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
... 12 more
2019-01-28 10:50:37,737 INFO [pool-13-thread-1] spark.SparkExecutable:38 :
2019-01-28 10:50:37 INFO ShutdownHookManager:54 - Shutdown hook called
Hubert
