[incubator-kyuubi] branch branch-1.5 updated: [KYUUBI #2060] Clear job group for init SQL

[ulyssesyou]

This is an automated email from the ASF dual-hosted git repository.

ulyssesyou pushed a commit to branch branch-1.5
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git


The following commit(s) were added to refs/heads/branch-1.5 by this push:
     new bd751df  [KYUUBI #2060] Clear job group for init SQL
bd751df is described below

commit bd751df828fc728b1555b84ea6d7daed5f30c02f
Author: Cheng Pan <cheng...@apache.org>
AuthorDate: Tue Mar 8 13:24:52 2022 +0800

    [KYUUBI #2060] Clear job group for init SQL
    
    <!--
    Thanks for sending a pull request!
    
    Here are some tips for you:
      1. If this is your first time, please read our contributor guidelines: 
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
      2. If the PR is related to an issue in 
https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your 
PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
      3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., 
'[WIP][KYUUBI #XXXX] Your PR title ...'.
    -->
    
    ### _Why are the changes needed?_
    <!--
    Please clarify why the changes are needed. For instance,
      1. If you add a feature, you can talk about the use case of it.
      2. If you fix a bug, you can clarify why it is a bug.
    -->
    We need to clear job group information for init SQL, otherwise in 
incremental collection mode, the fetchNext will always display wrong 
description.
    
    ### _How was this patch tested?_
    - [ ] Add some test cases that check the changes thoroughly including 
negative and positive cases if possible
    
    - [x] Add screenshots for manual tests if appropriate
    
    Before this patch:
    
    <img width="1916" alt="incremental_collect" 
src="https://user-images.githubusercontent.com/26535726/157165964-502ad809-9d99-44de-b08c-131d135ce33d.png";>
    
    After this patch:
    
    <img width="1917" alt="Xnip2022-03-08_12-04-50" 
src="https://user-images.githubusercontent.com/26535726/157165975-68eddf24-e0ad-4db2-931c-c75541802a7b.png";>
    
    - [ ] [Run 
test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests)
 locally before make a pull request
    
    Closes #2060 from pan3793/clear.
    
    Closes #2060
    
    76a67372 [Cheng Pan] Clear job group for init SQL
    
    Authored-by: Cheng Pan <cheng...@apache.org>
    Signed-off-by: ulysses-you <ulysses...@apache.org>
    (cherry picked from commit f8d9010ba57f7c4e12daf4626376edc7bc0421c0)
    Signed-off-by: ulysses-you <ulysses...@apache.org>
---
 .../main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala | 1 +
 .../kyuubi/engine/spark/session/SparkSQLSessionManager.scala       | 7 ++++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
index 31a2828..b6de175 100644
--- 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
+++ 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
@@ -133,6 +133,7 @@ object SparkSQLEngine extends Logging {
           interruptOnCancel = true)
         debug(s"Execute session initializing sql: $sqlStr")
         session.sql(sqlStr).isEmpty
+        session.sparkContext.clearJobGroup()
       }
     session
   }
diff --git 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/session/SparkSQLSessionManager.scala
 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/session/SparkSQLSessionManager.scala
index 4a8769b..fc039d8 100644
--- 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/session/SparkSQLSessionManager.scala
+++ 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/session/SparkSQLSessionManager.scala
@@ -66,8 +66,13 @@ class SparkSQLSessionManager private (name: String, spark: 
SparkSession)
         } else {
           val ss = spark.newSession()
           this.conf.get(ENGINE_SESSION_INITIALIZE_SQL).foreach { sqlStr =>
-            ss.sparkContext.setJobDescription(sqlStr)
+            ss.sparkContext.setJobGroup(
+              "engine_initializing_queries",
+              sqlStr,
+              interruptOnCancel = true)
+            debug(s"Execute session initializing sql: $sqlStr")
             ss.sql(sqlStr).isEmpty
+            ss.sparkContext.clearJobGroup()
           }
           ss
         }