[PR] [KYUUBI #6754] Improve the performance of ranger access requests [kyuubi]

[via GitHub]

wankunde opened a new pull request, #6758:
URL: https://github.com/apache/kyuubi/pull/6758

   # :mag: Description
   ## Issue References 🔗
   
   This pull request fixes #6754
   
   ## Describe Your Solution 🔧
   
   Right now in RuleAuthorization we use an ArrayBuffer to collect access 
requests, which is very slow because each new PrivilegeObject needs to be 
compared with all access requests.
   
   
   ## Types of changes :bookmark:
   
   - [x] Bugfix (non-breaking change which fixes an issue)
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to change)
   
   ## Test Plan 🧪
   
   #### Behavior Without This Pull Request :coffin:
   
   
![Before](https://github.com/user-attachments/assets/b8b76383-cced-4e8b-9116-9bf9694f32d6)
   
   #### Behavior With This Pull Request :tada:
   
   
![After](https://github.com/user-attachments/assets/25859dee-0f4c-4f85-9c55-82378121ad30)
   
   
   #### Related Unit Tests
   
   Test with local 50000 files:
   ```java
   test("KYUUBI #6754: improve the performance of ranger access requests") {
       val outputPath = 
"/private/var/folders/tr/scn8dgl13_l6_sh17bghtln1b35kn1/T/kyuubi-test-5492934124608743789/"
       println("output path: "+ outputPath)
   
       val plugin = mock[SparkRangerAdminPlugin.type]
       when(plugin.verify(Seq(any[RangerAccessRequest]), 
any[SparkRangerAuditHandler]))
         .thenAnswer(_ => ())
   
       val df = spark.read.parquet(outputPath + "/*/*.parquet")
       val plan = df.queryExecution.optimizedPlan
       val start = System.currentTimeMillis()
       RuleAuthorization(spark).checkPrivileges(spark, plan)
       val end = System.currentTimeMillis()
       println(s"Time elapsed : ${end - start} ms")
     }
   ```
   
   ---
   
   # Checklist 📝
   <!--- Go over all the following points, and put an `x` in all the boxes that 
apply. -->
   <!--- If you're unsure about any of these, don't hesitate to ask. We're here 
to help! -->
   
   - [ x] This patch was not authored or co-authored using [Generative 
Tooling](https://www.apache.org/legal/generative-tooling.html)
   
   **Be nice. Be informative.**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org
For additional commands, e-mail: notifications-h...@kyuubi.apache.org