surya created HIVE-27486:
----------------------------
Summary: Spark Thrift Server fails to start
Key: HIVE-27486
URL: https://issues.apache.org/jira/browse/HIVE-27486
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: surya
I am trying to use logback with spark-thrift-server to log the application logs
in elasticsearch and for that i am using *log4j-over-slf4j* along with
logback-classic and logback-core.
It seems Hive use WriterAppender.setWriter() of log4j for logging but the Log4j
bridge (log4j-over-slf4j library JAR) does not implement this method.
Getting this error:-
{code:java}
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hive.service.cli.operation.LogDivertAppender.setWriter(Ljava/io/Writer;)V
at
org.apache.hive.service.cli.operation.LogDivertAppender.<init>(LogDivertAppender.java:166)
at
org.apache.hive.service.cli.operation.OperationManager.initOperationLogCapture(OperationManager.java:84)
at
org.apache.hive.service.cli.operation.OperationManager.init(OperationManager.java:62)
at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
at
org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:83)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLSessionManager.init(SparkSQLSessionManager.scala:44)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.$anonfun$initCompositeService$1(SparkSQLCLIService.scala:123)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.$anonfun$initCompositeService$1$adapted(SparkSQLCLIService.scala:123)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.initCompositeService(SparkSQLCLIService.scala:123)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.initCompositeService$(SparkSQLCLIService.scala:120)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.initCompositeService(SparkSQLCLIService.scala:41)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:93)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.$anonfun$initCompositeService$1(SparkSQLCLIService.scala:123)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.$anonfun$initCompositeService$1$adapted(SparkSQLCLIService.scala:123)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.initCompositeService(SparkSQLCLIService.scala:123)
at
org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService.initCompositeService$(SparkSQLCLIService.scala:120)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.initCompositeService(HiveThriftServer2.scala:124)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.init(HiveThriftServer2.scala:144)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:63)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code}
i have reported same this on stackoverflow, for more details please refer:-
[https://stackoverflow.com/questions/76611022/spark-thrift-server-fails-to-start]
My question is how to resolve this issue, i searched online and people
suggesting to disable this
property[{*}hive.server2.logging.operation.enabled{*}] but disabling this
property won't work for me.
i need a solution so that i can use logback with hiveserver2/spark-thrift-server
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
