[kyuubi] branch master updated: [KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode

[chengpan]

This is an automated email from the ASF dual-hosted git repository.

chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git


The following commit(s) were added to refs/heads/master by this push:
     new 1dbb31b60 [KYUUBI #5148] Improve spark.driver.host assignment in Spark 
on K8s client mode
1dbb31b60 is described below

commit 1dbb31b601e9be2879c02fcfb46b1435f87df92d
Author: marcoluo <marco...@verizontal.com>
AuthorDate: Thu Aug 10 00:49:53 2023 +0800

    [KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client 
mode
    
    ### _Why are the changes needed?_
    
    Before this PR, the ip address obtained in the K8s environment is 
incorrect, and `spark.driver.host` cannot be manually specified.
    
    This time pr will adjust the way the IP is obtained and support the 
external parameter `spark.driver.host`
    
    ### _How was this patch tested?_
    
    It has been tested in the local environment and verified as expected
    
    Add screenshots for manual tests if appropriate
    
    
![image](https://github.com/apache/kyuubi/assets/29895551/5a0b821d-2194-441f-9635-d19f3d688057)
    
    Closes #5148 from fantasticKe/feat_k8s_driver_ip.
    
    Closes #5148
    
    330f5f93d [marcoluo] feat: 修改k8s模式下获取spark.driver.host的方式
    
    Authored-by: marcoluo <marco...@verizontal.com>
    Signed-off-by: Cheng Pan <cheng...@apache.org>
---
 .../src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
index 5badf695e..5f91bc73d 100644
--- 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
+++ 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
@@ -17,7 +17,6 @@
 
 package org.apache.kyuubi.engine.spark
 
-import java.net.InetAddress
 import java.time.Instant
 import java.util.{Locale, UUID}
 import java.util.concurrent.{CountDownLatch, ScheduledExecutorService, 
ThreadPoolExecutor, TimeUnit}
@@ -254,7 +253,7 @@ object SparkSQLEngine extends Logging {
 
       if (!isOnK8sClusterMode) {
         // set driver host to ip instead of kyuubi pod name
-        _sparkConf.set("spark.driver.host", 
InetAddress.getLocalHost.getHostAddress)
+        _sparkConf.setIfMissing("spark.driver.host", 
Utils.findLocalInetAddress.getHostAddress)
       }
     }