[
https://issues.apache.org/jira/browse/KYLIN-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
weibin0516 updated KYLIN-4255:
------------------------------
Description:
Currently, when using livy build error, the error message does not show the
detailed error reason, which is not conducive to troubleshooting. The following
is an example of submit build job error:
Current error message:
{code:java}
java.lang.RuntimeException: livy execute failed.
livy get status failed. state is dead
at
org.apache.kylin.common.livy.LivyRestExecutor.execute(LivyRestExecutor.java:76)
at
org.apache.kylin.source.hive.MRHiveDictUtil.runLivySqlJob(MRHiveDictUtil.java:144)
at
org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.createFlatHiveTable(CreateFlatHiveTableByLivyStep.java:44)
at
org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.doWork(CreateFlatHiveTableByLivyStep.java:51)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
Actual reason for the error:
{code:java}
2019-11-13 07:40:02 WARN NativeCodeLoader:62 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.io.FileNotFoundException: File
hdfs://localhost:9000/kylin/livy/hbase-client-$HBASE_VERSION.jar does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:697)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:105)
at
org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:755)
at
org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:751)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:751)
at org.apache.spark.util.Utils$.fetchHcfsFile(Utils.scala:727)
at org.apache.spark.util.Utils$.doFetchFile(Utils.scala:695)
at
org.apache.spark.deploy.DependencyUtils$.downloadFile(DependencyUtils.scala:135)
at
org.apache.spark.deploy.DependencyUtils$$anonfun$downloadFileList$2.apply(DependencyUtils.scala:102)
at
org.apache.spark.deploy.DependencyUtils$$anonfun$downloadFileList$2.apply(DependencyUtils.scala:102)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
{code}
> Display detailed error message when using livy build error
> ----------------------------------------------------------
>
> Key: KYLIN-4255
> URL: https://issues.apache.org/jira/browse/KYLIN-4255
> Project: Kylin
> Issue Type: Improvement
> Components: Spark Engine
> Reporter: weibin0516
> Assignee: weibin0516
> Priority: Major
>
> Currently, when using livy build error, the error message does not show the
> detailed error reason, which is not conducive to troubleshooting. The
> following is an example of submit build job error:
> Current error message:
> {code:java}
> java.lang.RuntimeException: livy execute failed.
> livy get status failed. state is dead
> at
> org.apache.kylin.common.livy.LivyRestExecutor.execute(LivyRestExecutor.java:76)
> at
> org.apache.kylin.source.hive.MRHiveDictUtil.runLivySqlJob(MRHiveDictUtil.java:144)
> at
> org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.createFlatHiveTable(CreateFlatHiveTableByLivyStep.java:44)
> at
> org.apache.kylin.source.hive.CreateFlatHiveTableByLivyStep.doWork(CreateFlatHiveTableByLivyStep.java:51)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Actual reason for the error:
> {code:java}
> 2019-11-13 07:40:02 WARN NativeCodeLoader:62 - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Exception in thread "main" java.io.FileNotFoundException: File
> hdfs://localhost:9000/kylin/livy/hbase-client-$HBASE_VERSION.jar does not
> exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:697)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:105)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:755)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:751)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:751)
> at org.apache.spark.util.Utils$.fetchHcfsFile(Utils.scala:727)
> at org.apache.spark.util.Utils$.doFetchFile(Utils.scala:695)
> at
> org.apache.spark.deploy.DependencyUtils$.downloadFile(DependencyUtils.scala:135)
> at
> org.apache.spark.deploy.DependencyUtils$$anonfun$downloadFileList$2.apply(DependencyUtils.scala:102)
> at
> org.apache.spark.deploy.DependencyUtils$$anonfun$downloadFileList$2.apply(DependencyUtils.scala:102)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
