Marco Gaido created LIVY-572:
--------------------------------
Summary: Log operations fails because of dependencies on Spark
Key: LIVY-572
URL: https://issues.apache.org/jira/browse/LIVY-572
Project: Livy
Issue Type: Bug
Components: Thriftserver
Reporter: Marco Gaido
When {{livy.server.thrift.logging.operation.enabled}} is {{true}}, which is by
default, and Spark jars are not available in Livy JVM, all operations fail due
to:
{{code}}
19/03/23 10:56:35 INFO ThriftBinaryCLIService: Closing the session:
SessionHandle [eaf96260-ef2b-438e-9d12-3812f1095201]
Exception in thread "LivyThriftserver-Handler-Pool: Thread-30"
java.lang.NoClassDefFoundError: org/apache/spark/sql/Row
at
org.apache.livy.thriftserver.session.ColumnBuffer.toHiveString(ColumnBuffer.java:296)
at
org.apache.livy.thriftserver.session.ColumnBuffer.add(ColumnBuffer.java:183)
at
org.apache.livy.thriftserver.serde.ColumnOrientedResultSet.addRow(ThriftResultSet.scala:108)
at
org.apache.livy.thriftserver.LivyOperationManager$$anonfun$getOperationLogRowSet$2.apply(LivyOperationManager.scala:125)
at
org.apache.livy.thriftserver.LivyOperationManager$$anonfun$getOperationLogRowSet$2.apply(LivyOperationManager.scala:124)
at scala.collection.immutable.List.foreach(List.scala:392)
at
scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
at
org.apache.livy.thriftserver.LivyOperationManager.getOperationLogRowSet(LivyOperationManager.scala:124)
at
org.apache.livy.thriftserver.LivyOperationManager.fetchResults(LivyOperationManager.scala:229)
at
org.apache.livy.thriftserver.LivyCLIService.fetchResults(LivyCLIService.scala:353)
at
org.apache.livy.thriftserver.cli.ThriftCLIService.FetchResults(ThriftCLIService.scala:609)
at
org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1837)
at
org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1822)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.Row
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 21 more
{{code}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
