[ 
https://issues.apache.org/jira/browse/LIVY-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marco Gaido reassigned LIVY-571:
--------------------------------

    Assignee: Jeffrey(Xilang) Yan

> When an exception happens running a query, we should report it to end user
> --------------------------------------------------------------------------
>
>                 Key: LIVY-571
>                 URL: https://issues.apache.org/jira/browse/LIVY-571
>             Project: Livy
>          Issue Type: Bug
>          Components: Thriftserver
>            Reporter: Marco Gaido
>            Assignee: Jeffrey(Xilang) Yan
>            Priority: Major
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> When a query fails with an exception on livy thriftserver, instead of 
> reporting this exception to the end user, a meaningless one is reported. Eg, 
> with Hive support not enabled on spark, the following query causes:
> {code}
> 0: jdbc:hive2://localhost:10090/> create table test as select a.* from 
> (select 1, "2") a;
> Error: java.lang.RuntimeException: java.util.NoSuchElementException: 
> Statement 820bb5c2-018b-46ea-9b7f-b0e3b9c31c46 not found in session 
> acf3712b-1f08-4111-950f-559fc3f3f10c.
> org.apache.livy.thriftserver.session.ThriftSessionState.statementNotFound(ThriftSessionState.java:118)
> org.apache.livy.thriftserver.session.ThriftSessionState.cleanupStatement(ThriftSessionState.java:107)
> org.apache.livy.thriftserver.session.CleanupStatementJob.call(CleanupStatementJob.java:43)
> org.apache.livy.thriftserver.session.CleanupStatementJob.call(CleanupStatementJob.java:26)
> org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:64)
> org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:31)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
> {code}
> Looking at the logs, of course the real problem is:
> {code}
> 19/03/23 10:40:32 ERROR LivyExecuteStatementOperation: Error running hive 
> query: 
> org.apache.hive.service.cli.HiveSQLException: 
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
> org.apache.spark.sql.AnalysisException: Hive support is required to CREATE 
> Hive TABLE (AS SELECT);;
> 'CreateTable `test`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, 
> ErrorIfExists
> +- Project [1#1, 2#2]
>    +- SubqueryAlias `a`
>       +- Project [1 AS 1#1, 2 AS 2#2]
>          +- OneRowRelation
> org.apache.spark.sql.execution.datasources.HiveOnlyCheck$$anonfun$apply$12.apply(rules.scala:392)
> org.apache.spark.sql.execution.datasources.HiveOnlyCheck$$anonfun$apply$12.apply(rules.scala:390)
> org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:117)
> org.apache.spark.sql.execution.datasources.HiveOnlyCheck$.apply(rules.scala:390)
> org.apache.spark.sql.execution.datasources.HiveOnlyCheck$.apply(rules.scala:388)
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$2.apply(CheckAnalysis.scala:386)
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$2.apply(CheckAnalysis.scala:386)
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:386)
> org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:95)
> org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$executeAndCheck$1.apply(Analyzer.scala:108)
> org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$executeAndCheck$1.apply(Analyzer.scala:105)
> org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:201)
> org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:105)
> org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
> org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
> org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
> org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
> org.apache.livy.thriftserver.session.SqlJob.executeSql(SqlJob.java:74)
> org.apache.livy.thriftserver.session.SqlJob.call(SqlJob.java:64)
> org.apache.livy.thriftserver.session.SqlJob.call(SqlJob.java:35)
> org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:64)
> org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:31)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:748)
>       at 
> org.apache.livy.thriftserver.LivyExecuteStatementOperation.execute(LivyExecuteStatementOperation.scala:147)
>       at 
> org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$1$$anon$2.run(LivyExecuteStatementOperation.scala:97)
>       at 
> org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$1$$anon$2.run(LivyExecuteStatementOperation.scala:94)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:422)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
>       at 
> org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$1.run(LivyExecuteStatementOperation.scala:107)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>       at java.lang.Thread.run(Thread.java:748)
>  ....
> {code}
> And this should be reported to the end user.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Reply via email to