(kyuubi) branch master updated: [KYUUBI #6463] Release semaphore immediately after startup process exit

[chengpan]

This is an automated email from the ASF dual-hosted git repository.

chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git


The following commit(s) were added to refs/heads/master by this push:
     new 95ed74821 [KYUUBI #6463] Release semaphore immediately after startup 
process exit
95ed74821 is described below

commit 95ed74821c3a2d3a3f402033de7b463966a4bc28
Author: ic4y <i...@apache.org>
AuthorDate: Thu Jun 13 21:10:21 2024 +0800

    [KYUUBI #6463] Release semaphore immediately after startup process exit
    
    # :mag: Description
    ## Issue References 🔗
    
    The concurrency limit for the engine startup process is mainly used to 
avoid overload on the machine(or container) of the Kyuubi server, the current 
implementation holds startupProcessSemaphore until the session is established 
successfully. While for Spark on YARN cluster mode, some YARN queue resource 
insufficiency may block the subsequent Spark application submissions to other 
queues, significantly affecting the Kyuubi server's resource utilization.
    
    ## Describe Your Solution 🔧
    
    We should immediately release the `startupProcessSemaphore` after the 
engine startup process exits (i.e., after the `spark-submit` process exits) as 
the load has already disappeared.
    
    ## Types of changes :bookmark:
    
    - [x] Bugfix (non-breaking change which fixes an issue)
    - [ ] New feature (non-breaking change which adds functionality)
    - [ ] Breaking change (fix or feature that would cause existing 
functionality to change)
    
    ## Test Plan 🧪
    
    I tested it on a cluster of 50 kyuubi Servers, and kyuubi server resource 
utilization increased by 70%
    
    ---
    
    # Checklist 📝
    
    - [ ] This patch was not authored or co-authored using [Generative 
Tooling](https://www.apache.org/legal/generative-tooling.html)
    
    **Be nice. Be informative.**
    
    Closes #6463 from ic4y/master-p003.
    
    Closes #6463
    
    f7de68ce3 [ic4y] Improve code quality
    d8b0248df [ic4y] [Improve][EngineRef] Optimize Engine Startup Concurrency 
Limit
    
    Authored-by: ic4y <i...@apache.org>
    Signed-off-by: Cheng Pan <cheng...@apache.org>
---
 .../src/main/scala/org/apache/kyuubi/engine/EngineRef.scala          | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git 
a/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/EngineRef.scala 
b/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/EngineRef.scala
index bb7f7ecbc..30bf32c39 100644
--- a/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/EngineRef.scala
+++ b/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/EngineRef.scala
@@ -239,7 +239,10 @@ private[kyuubi] class EngineRef(
       while (engineRef.isEmpty) {
         if (exitValue.isEmpty && process.waitFor(1, TimeUnit.SECONDS)) {
           exitValue = Some(process.exitValue())
-          if (!exitValue.contains(0)) {
+          if (exitValue.contains(0)) {
+            acquiredPermit = false
+            startupProcessSemaphore.foreach(_.release())
+          } else {
             val error = builder.getError
             MetricsSystem.tracing { ms =>
               ms.incCount(MetricRegistry.name(ENGINE_FAIL, appUser))