(kyuubi) branch branch-1.9 updated: [KYUUBI #6302] Call cancelJobGroup immediately after statement execution finished

[chengpan]

This is an automated email from the ASF dual-hosted git repository.

chengpan pushed a commit to branch branch-1.9
in repository https://gitbox.apache.org/repos/asf/kyuubi.git


The following commit(s) were added to refs/heads/branch-1.9 by this push:
     new 686ae2046 [KYUUBI #6302] Call cancelJobGroup immediately after 
statement execution finished
686ae2046 is described below

commit 686ae204613d30460cfc0eb0e6668c465ce2d4d3
Author: xorsum <xor...@outlook.com>
AuthorDate: Mon Jun 17 18:39:08 2024 +0800

    [KYUUBI #6302] Call cancelJobGroup immediately after statement execution 
finished
    
    # :mag: Description
    ## Issue References 🔗
    
    This pull request fixes #6302 : when the SQL is submitted by Kyuubi Rest 
Api (the same kyuubi path with BEELINE), the legacy job will not be cancelled, 
https://github.com/apache/kyuubi/issues/6302#issuecomment-2160572624 .
    
    The reason:
        Beeline session calls `cancelJobGroup` in `SparkOperation#cleanup`.
        But  Rest Api session doesn't call `SparkOperation#cleanup`, so the 
legacy job submitted by Rest Api will not be canceled.
    
    ## Describe Your Solution 🔧
    
    The modification:
        call `SparkOperation#cleanup` in `ExecuteStatement#executeStatement`'s 
finally clause.
    
    ## Types of changes :bookmark:
    
    - [x] Bugfix (non-breaking change which fixes an issue)
    - [ ] New feature (non-breaking change which adds functionality)
    - [ ] Breaking change (fix or feature that would cause existing 
functionality to change)
    
    ## Test Plan 🧪
    
    #### Behavior Without This Pull Request :coffin:
    
    #### Behavior With This Pull Request :tada:
    
    #### Related Unit Tests
    
    ---
    
    # Checklist 📝
    
    - [x] This patch was not authored or co-authored using [Generative 
Tooling](https://www.apache.org/legal/generative-tooling.html)
    
    **Be nice. Be informative.**
    
    Closes #6473 from XorSum/features/cancel-after-execute-stmt.
    
    Closes #6302
    
    16dd508e4 [xorsum] operation executeStatement cancel group
    
    Authored-by: xorsum <xor...@outlook.com>
    Signed-off-by: Cheng Pan <cheng...@apache.org>
    (cherry picked from commit 7de6371d9af08c6833e0c8d88972a552f940e0c1)
    Signed-off-by: Cheng Pan <cheng...@apache.org>
---
 .../org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala      | 1 +
 1 file changed, 1 insertion(+)

diff --git 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
index f6eccb100..90bfe8336 100644
--- 
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
+++ 
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
@@ -104,6 +104,7 @@ class ExecuteStatement(
       onError(cancel = true)
     } finally {
       shutdownTimeoutMonitor()
+      if (!spark.sparkContext.isStopped) 
spark.sparkContext.cancelJobGroup(statementId)
     }
 
   override protected def runInternal(): Unit = {