inkkim opened a new issue, #6799: URL: https://github.com/apache/kyuubi/issues/6799
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug org.apache.kyuubi.KyuubiSQLException: org.apache.kyuubi.KyuubiSQLException: Exception in thread "main" java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1894) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:174) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1120) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1129) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: org.apache.kyuubi.KyuubiException: Failed to initialize SparkSQLEngine: org.apache.kyuubi.util.JavaUtils.findLocalInetAddress()Ljava/net/InetAddress; at org.apache.kyuubi.engine.spark.SparkSQLEngine$.$anonfun$startEngine$1(SparkSQLEngine.scala:357) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.$anonfun$startEngine$1$adapted(SparkSQLEngine.scala:342) at scala.Option.foreach(Option.scala:407) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.startEngine(SparkSQLEngine.scala:342) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.main(SparkSQLEngine.scala:407) at org.apache.kyuubi.engine.spark.SparkSQLEngine.main(SparkSQLEngine.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) ### Affects Version(s) v1.10 ### Kyuubi Server Log Output ```logtalk 2024-11-06 18:35:04.034 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.shaded.curator.framework.imps.CuratorFrameworkImpl: Starting 2024-11-06 18:35:04.034 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.shaded.zookeeper.ZooKeeper: Initiating client connection, connectString=ZOOKE EPER-001:2181,ZOOKEEPER-002:2181,ZOOKEEPER-003:2181 sessionTimeout=60000 watcher=org.apache.kyuubi.shaded.curator.ConnectionState@1e19cc16 2024-11-06 18:35:04.035 INFO KyuubiSessionManager-exec-pool: Thread-219-SendThread(ZOOKEEPER-003:2181) org.apache.kyuubi.shaded.zookeeper.ClientCnxn: Opening socket co nnection to server ZOOKEEPER-003/10.10.10.10:2181. Will not attempt to authenticate using SASL (unknown error) 2024-11-06 18:35:04.036 INFO KyuubiSessionManager-exec-pool: Thread-219-SendThread(ZOOKEEPER-003:2181) org.apache.kyuubi.shaded.zookeeper.ClientCnxn: Socket connection established to ZOOKEEPER-003/10.10.10.10:2181, initiating session 2024-11-06 18:35:04.037 INFO KyuubiSessionManager-exec-pool: Thread-219-SendThread(ZOOKEEPER-003:2181) org.apache.kyuubi.shaded.zookeeper.ClientCnxn: Session establish ment complete on server ZOOKEEPER-003/10.10.10.10:2181, sessionid = 0x300231d79b100b7, negotiated timeout = 40000 2024-11-06 18:35:04.038 INFO KyuubiSessionManager-exec-pool: Thread-219-EventThread org.apache.kyuubi.shaded.curator.framework.state.ConnectionStateManager: State change: CONNECTED 2024-11-06 18:35:04.044 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.Utils: Loading Kyuubi properties from /usr/local/spark/conf/spark-defaults.conf 2024-11-06 18:35:04.046 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.engine.ProcBuilder: Logging to /opt/apache-kyuubi-1.10.0-bin/work/kyuubi/kyuubi-spark-sql -engine.log.10 2024-11-06 18:35:04.047 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.engine.EngineRef: Launching engine: /opt/spark-3.5.1-bin-hadoop3/bin/spark-submit \ --class org.apache.kyuubi.engine.spark.SparkSQLEngine \ --conf spark.hive.server2.thrift.resultset.default.fetch.size=1000 \ --conf spark.kyuubi.client.ipAddress=127.0.0.1 \ --conf spark.kyuubi.client.version=1.10.0 \ --conf spark.kyuubi.engine.appMgrInfo=eyJyZXNvdXJjZU1hbmFnZXIiOiJ5YXJuIiwia3ViZXJuZXRlc0luZm8iOnsiY29udGV4dCI6bnVsbCwibmFtZXNwYWNlIjpudWxsfX0= \ --conf spark.kyuubi.engine.engineLog.path=/opt/apache-kyuubi-1.10.0-bin/work/kyuubi/kyuubi-spark-sql-engine.log.10 \ --conf spark.kyuubi.engine.share.level=USER \ --conf spark.kyuubi.engine.submit.time=1730885704044 \ --conf spark.kyuubi.engine.type=SPARK_SQL \ --conf spark.kyuubi.engine.ui.stop.enabled=true \ --conf spark.kyuubi.frontend.protocols=THRIFT_BINARY,REST \ --conf spark.kyuubi.ha.addresses=ZOOKEEPER-001:2181,ZOOKEEPER-002:2181,ZOOKEEPER-003:2181 \ --conf spark.kyuubi.ha.engine.ref.id=66be7e12-49dc-4de1-890f-297c086342a8 \ --conf spark.kyuubi.ha.namespace=/kyuubi_1.10.0_USER_SPARK_SQL/kyuubi/default \ --conf spark.kyuubi.operation.result.format=arrow \ --conf spark.kyuubi.server.ipAddress=10.10.10.10 \ --conf spark.kyuubi.session.connection.url=10.10.10.10:10009 \ --conf spark.kyuubi.session.engine.idle.timeout=PT0M \ --conf spark.kyuubi.session.engine.spark.showProgress=true \ --conf spark.kyuubi.session.real.user=kyuubi \ --conf spark.app.name=kyuubi_USER_SPARK_SQL_kyuubi_default_66be7e12-49dc-4de1-890f-297c086342a8 \ --conf spark.driver.extraJavaOptions=-Daws.region=us-west-2 \ --conf spark.executor.extraJavaOptions=-Daws.region=us-west-2 \ --conf spark.sql.catalog.sample=org.apache.iceberg.spark.SparkCatalog \ --conf spark.sql.catalog.sample.io-impl=org.apache.iceberg.aws.s3.S3FileIO \ --conf spark.sql.catalog.sample.s3.access-key-id=sample \ --conf spark.sql.catalog.sample.s3.endpoint=http://10.10.10.10:9020 \ --conf spark.sql.catalog.sample.s3.path-style-access=true \ --conf spark.sql.catalog.sample.s3.secret-access-key=********************** \ --conf spark.sql.catalog.sample.type=rest \ --conf spark.sql.catalog.sample.uri=http://rest.test.com:9083 \ --conf spark.sql.catalog.sample.warehouse=s3a://sample/ \ --conf spark.sql.defaultCatalog=sample \ --conf spark.yarn.maxAppAttempts=1 \ --conf spark.yarn.tags=KYUUBI,66be7e12-49dc-4de1-890f-297c086342a8 \ --proxy-user kyuubi /opt/apache-kyuubi-1.10.0-bin/externals/engines/spark/kyuubi-spark-sql-engine_2.12-1.10.0.jar 2024-11-06 18:35:39.450 INFO Curator-Framework-0 org.apache.kyuubi.shaded.curator.framework.imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting 2024-11-06 18:35:39.452 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.shaded.zookeeper.ZooKeeper: Session: 0x300231d79b100b7 closed 2024-11-06 18:35:39.452 INFO KyuubiSessionManager-exec-pool: Thread-219-EventThread org.apache.kyuubi.shaded.zookeeper.ClientCnxn: EventThread shut down for session: 0x300231d79b100b7 2024-11-06 18:35:39.453 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.operation.LaunchEngine: Processing kyuubi's query[c4c29906-fd8b-47d5-8228-b18dcf6a67d6]: RUNNING_STATE -> ERROR_STATE, time taken: 35.419 seconds 2024-11-06 18:35:39.453 INFO KyuubiSessionManager-exec-pool: Thread-219 org.apache.kyuubi.operation.OperationAuditLogger: operation=c4c29906-fd8b-47d5-8228-b18dcf6a67d6 opType=LaunchEngine state=ERROR user=kyuubi session=66be7e12-49dc-4de1-890f-297c086342a8 2024-11-06 18:35:39.741 INFO KyuubiTBinaryFrontendHandler-Pool: Thread-206 org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Received request of closing SessionHandle [66be7e12-49dc-4de1-890f-297c086342a8] 2024-11-06 18:35:39.741 INFO KyuubiTBinaryFrontendHandler-Pool: Thread-206 org.apache.kyuubi.session.KyuubiSessionManager: kyuubi's KyuubiSessionImpl with SessionHandle [66be7e12-49dc-4de1-890f-297c086342a8] is closed, current opening sessions 0 2024-11-06 18:35:39.741 INFO KyuubiTBinaryFrontendHandler-Pool: Thread-206 org.apache.kyuubi.operation.LaunchEngine: Processing kyuubi's query[c4c29906-fd8b-47d5-8228-b18dcf6a67d6]: ERROR_STATE -> CLOSED_STATE, time taken: 35.707 seconds 2024-11-06 18:35:39.741 INFO KyuubiTBinaryFrontendHandler-Pool: Thread-206 org.apache.kyuubi.operation.OperationAuditLogger: operation=c4c29906-fd8b-47d5-8228-b18dcf6a67d6 opType=LaunchEngine state=CLOSED user=kyuubi session=66be7e12-49dc-4de1-890f-297c086342a8 2024-11-06 18:35:39.742 INFO KyuubiTBinaryFrontendHandler-Pool: Thread-206 org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Finished closing SessionHandle [66be7e12-49dc-4de1-890f-297c086342a8] 2024-11-06 18:36:44.329 ERROR SIGTERM handler org.apache.kyuubi.server.KyuubiServer: RECEIVED SIGNAL 15: TERM 2024-11-06 18:36:44.332 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [KyuubiRestFrontendService] is stopping. 2024-11-06 18:36:44.335 INFO shutdown-hook-0 org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@34780cd9{HTTP/1.1, (http/1.1)}{10.10.10.10:10099} 2024-11-06 18:36:44.337 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiRestFrontendService: Service[KyuubiRestFrontendService] is stopped. 2024-11-06 18:36:44.337 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [KyuubiTBinaryFrontend] is stopping. 2024-11-06 18:36:44.337 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Service: [KyuubiServiceDiscovery] is stopping. 2024-11-06 18:36:44.339 INFO Curator-Framework-0 org.apache.kyuubi.shaded.curator.framework.imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting 2024-11-06 18:36:44.339 WARN main-EventThread org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient: This Kyuubi instance 10.10.10.10:10009 is now de-registered from ZooKeeper. The server will be shut down after the last client session completes. 2024-11-06 18:36:44.340 INFO shutdown-hook-0 org.apache.kyuubi.shaded.zookeeper.ZooKeeper: Session: 0x300231d79b1009c closed 2024-11-06 18:36:44.340 INFO shutdown-hook-0 org.apache.kyuubi.ha.client.KyuubiServiceDiscovery: Service[KyuubiServiceDiscovery] is stopped. 2024-11-06 18:36:44.340 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Service[KyuubiTBinaryFrontend] is stopped. 2024-11-06 18:36:44.341 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiTBinaryFrontendService: KyuubiTBinaryFrontend has stopped 2024-11-06 18:36:44.341 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [KyuubiBackendService] is stopping. 2024-11-06 18:36:44.341 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [KyuubiSessionManager] is stopping. 2024-11-06 18:36:44.341 INFO shutdown-hook-0 org.apache.kyuubi.session.KyuubiSessionManager: Service: [KyuubiOperationManager] is stopping. 2024-11-06 18:36:44.341 INFO shutdown-hook-0 org.apache.kyuubi.operation.KyuubiOperationManager: Service[KyuubiOperationManager] is stopped. 2024-11-06 18:36:44.341 INFO shutdown-hook-0 org.apache.kyuubi.session.KyuubiSessionManager: Service: [MetadataManager] is stopping. 2024-11-06 18:36:44.342 INFO shutdown-hook-0 com.zaxxer.hikari.HikariDataSource: jdbc-metadata-store-pool - Shutdown initiated... 2024-11-06 18:36:44.344 INFO shutdown-hook-0 com.zaxxer.hikari.HikariDataSource: jdbc-metadata-store-pool - Shutdown completed. 2024-11-06 18:36:44.344 INFO shutdown-hook-0 org.apache.kyuubi.server.metadata.MetadataManager: Service[MetadataManager] is stopped. 2024-11-06 18:36:44.344 INFO shutdown-hook-0 org.apache.kyuubi.session.KyuubiSessionManager: Service: [TempFileService] is stopping. 2024-11-06 18:36:44.345 INFO shutdown-hook-0 org.apache.kyuubi.service.TempFileService: Service[TempFileService] is stopped. 2024-11-06 18:36:44.345 INFO shutdown-hook-0 org.apache.kyuubi.session.KyuubiSessionManager: Service: [HadoopCredentialsManager] is stopping. 2024-11-06 18:36:44.346 INFO shutdown-hook-0 org.apache.kyuubi.credentials.HadoopCredentialsManager: Service[HadoopCredentialsManager] is stopped. 2024-11-06 18:36:44.346 INFO shutdown-hook-0 org.apache.kyuubi.session.KyuubiSessionManager: Service: [KyuubiApplicationManager] is stopping. 2024-11-06 18:36:44.347 INFO shutdown-hook-0 org.apache.kyuubi.engine.KyuubiApplicationManager: Service[KyuubiApplicationManager] is stopped. 2024-11-06 18:36:44.347 INFO shutdown-hook-0 org.apache.kyuubi.session.KyuubiSessionManager: Service[KyuubiSessionManager] is stopped. 2024-11-06 18:36:44.352 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service[KyuubiBackendService] is stopped. 2024-11-06 18:36:44.352 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [MetricsSystem] is stopping. 2024-11-06 18:36:44.353 INFO shutdown-hook-0 org.apache.kyuubi.metrics.MetricsSystem: Service: [PrometheusReporterService] is stopping. 2024-11-06 18:36:44.353 INFO shutdown-hook-0 org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@54d18072{HTTP/1.1, (http/1.1)}{0.0.0.0:10019} 2024-11-06 18:36:44.354 INFO shutdown-hook-0 org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@712cfb63{/,null,STOPPED} 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.metrics.PrometheusReporterService: Prometheus metrics HTTP server has stopped. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.metrics.PrometheusReporterService: Service[PrometheusReporterService] is stopped. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.metrics.MetricsSystem: Service[MetricsSystem] is stopped. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [PeriodicGCService] is stopping. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.server.PeriodicGCService: Service[PeriodicGCService] is stopped. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service: [KinitAuxiliaryService] is stopping. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.server.KinitAuxiliaryService: Service[KinitAuxiliaryService] is stopped. 2024-11-06 18:36:44.355 INFO shutdown-hook-0 org.apache.kyuubi.server.KyuubiServer: Service[KyuubiServer] is stopped. 2024-11-06 18:36:44.355 INFO main-EventThread org.apache.kyuubi.shaded.zookeeper.ClientCnxn: EventThread shut down for session: 0x300231d79b1009c ``` ### Kyuubi Engine Log Output ```logtalk SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1894) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:174) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1120) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1129) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: org.apache.kyuubi.KyuubiException: Failed to initialize SparkSQLEngine: org.apache.kyuubi.util.JavaUtils.findLocalInetAddress()Ljava/net/InetAddress; at org.apache.kyuubi.engine.spark.SparkSQLEngine$.$anonfun$startEngine$1(SparkSQLEngine.scala:357) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.$anonfun$startEngine$1$adapted(SparkSQLEngine.scala:342) at scala.Option.foreach(Option.scala:407) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.startEngine(SparkSQLEngine.scala:342) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.main(SparkSQLEngine.scala:407) at org.apache.kyuubi.engine.spark.SparkSQLEngine.main(SparkSQLEngine.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1029) at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:176) at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) ... 6 more Caused by: java.lang.NoSuchMethodError: org.apache.kyuubi.util.JavaUtils.findLocalInetAddress()Ljava/net/InetAddress; at org.apache.kyuubi.service.TFrontendService.$anonfun$serverAddr$2(TFrontendService.scala:56) at scala.Option.getOrElse(Option.scala:189) at org.apache.kyuubi.service.TFrontendService.serverAddr$lzycompute(TFrontendService.scala:56) at org.apache.kyuubi.service.TFrontendService.serverAddr(TFrontendService.scala:55) at org.apache.kyuubi.service.TBinaryFrontendService.initialize(TBinaryFrontendService.scala:121) at org.apache.kyuubi.service.CompositeService.$anonfun$initialize$1(CompositeService.scala:40) at org.apache.kyuubi.service.CompositeService.$anonfun$initialize$1$adapted(CompositeService.scala:40) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.apache.kyuubi.service.CompositeService.initialize(CompositeService.scala:40) at org.apache.kyuubi.service.Serverable.initialize(Serverable.scala:48) at org.apache.kyuubi.engine.spark.SparkSQLEngine.initialize(SparkSQLEngine.scala:70) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.$anonfun$startEngine$1(SparkSQLEngine.scala:353) ... 22 more ``` ### Kyuubi Server Configurations ```yaml kyuubi.authentication NONE kyuubi.frontend.bind.host SPARKYARN-001 kyuubi.frontend.protocols THRIFT_BINARY,REST kyuubi.frontend.thrift.binary.bind.port 10009 kyuubi.frontend.rest.bind.port 10099 kyuubi.operation.result.format arrow kyuubi.engine.type SPARK_SQL kyuubi.engine.share.level USER kyuubi.session.engine.idle.timeout PT0M kyuubi.engine.ui.stop.enabled true kyuubi.session.engine.spark.showProgress true kyuubi.ha.addresses ZOOKEEPER-001:2181,ZOOKEEPER-002:2181,ZOOKEEPER-003:2181 kyuubi.ha.namespace kyuubi # kyuubi-env.sh export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 export SPARK_HOME=/usr/local/spark export HADOOP_HOME=/usr/local/hadoop export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin export KYUUBI_HOME=/opt/apache-kyuubi-1.10.0-bin export KYUUBI_ENGINE_MEMORY=4g export KYUUBI_JAVA_OPTS="-Xmx14g -Xms2g -XX:+UnlockDiagnosticVMOptions -XX:ParGCCardsPerStrideChunk=4096 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseCondCardMark -XX:MaxDirectMemorySize=4084m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./logs -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -Xloggc:./logs/kyuubi-server-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=5M -XX:NewRatio=3 -XX:MetaspaceSize=1024m" ``` ### Kyuubi Engine Configurations ```yaml spark.master yarn spark.executor.cores 3 spark.serializer org.apache.spark.serializer.KryoSerializer spark.driver.memory 3g spark.executor.memory 10g spark.yarn.am.memory 1g spark.eventLog.enabled true spark.eventLog.dir hdfs:///user/spark/history/done spark.history.fs.logDirectory hdfs:///user/spark/history/done spark.worker.cleanup.enabled true spark.worker.cleanup.interval 86400 spark.worker.cleanup.appDataTtl 604800 #spark.dynamicAllocation.enabled false spark.yarn.queue q1 spark.yarn.jars file:/usr/local/spark/jars/* #spark.history.fs.logDirectory file:/usr/local/spark/logs/spark-events spark.history.fs.logDirectory hdfs:///user/spark/history/done spark.eventLog.compress true spark.driver.extraClassPath=$SPARK_HOME/jars/spark-network-yarn_2.12-3.5.1.jar spark.submit.proxyUser.allowCustomClasspathInClusterMode true spark.shuffle.compress true spark.shuffle.spill.compress true spark.sql.shuffle.partitions 240 spark.rdd.compress true ``` ### Additional context Here’s a detailed issue description based on the provided information: --- ### **Title** Error Running Kyuubi Engine: Failed to Initialize SparkSQLEngine due to `KyuubiException` in `JavaUtils.findLocalInetAddress()` ### **Description** When attempting to start the Kyuubi Engine using the `kyuubi-hive-jdbc-shaded-1.10.0` driver to establish a JDBC Connection, the following error occurs: The Kyuubi Engine initiates a Spark application on YARN, but it fails shortly after with the following log output: ``` SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1894) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:174) ... Caused by: org.apache.kyuubi.KyuubiException: Failed to initialize SparkSQLEngine: org.apache.kyuubi.util.JavaUtils.findLocalInetAddress()Ljava/net/InetAddress; at org.apache.kyuubi.engine.spark.SparkSQLEngine$.$anonfun$startEngine$1(SparkSQLEngine.scala:357) ... Caused by: java.lang.NoSuchMethodError: org.apache.kyuubi.util.JavaUtils.findLocalInetAddress()Ljava/net/InetAddress; at org.apache.kyuubi.service.TFrontendService.$anonfun$serverAddr$2(TFrontendService.scala:56) ... ``` ### **Steps to Reproduce** 1. Start Kyuubi Engine using `kyuubi-hive-jdbc-shaded-1.10.0` driver to establish a JDBC Connection. 2. Observe that a Spark Application is created in YARN. 3. The process fails, generating the above log output and terminating with an error. ### **Observed Behavior** - The engine fails to initialize `SparkSQLEngine`, throwing a `KyuubiException` caused by a `NoSuchMethodError` in `org.apache.kyuubi.util.JavaUtils.findLocalInetAddress()`. ### **Expected Behavior** - Kyuubi Engine should initialize successfully and handle JDBC connections without termination. ### **Environment** - **Driver**: `kyuubi-hive-jdbc-shaded-1.10.0` - **Platform**: YARN, Spark ### **Additional Information** This issue might be related to missing or incompatible dependencies in `org.apache.kyuubi.util`. Any guidance on resolving this error or additional dependencies would be appreciated. --- This template should help provide a clear and structured description for others to understand and investigate the issue. ### Are you willing to submit PR? - [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. - [X] No. I cannot submit a PR at this time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org For additional commands, e-mail: notifications-h...@kyuubi.apache.org
