(spark) branch master updated: [SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness of `ReattachableExecuteSuite`

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 982c7268f4c [SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness 
of `ReattachableExecuteSuite`
982c7268f4c is described below

commit 982c7268f4c7ea1fa03ea679146f9e83f31bece7
Author: Juliusz Sompolski <ju...@databricks.com>
AuthorDate: Wed Dec 6 10:57:45 2023 -0800

    [SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness of 
`ReattachableExecuteSuite`
    
    ### What changes were proposed in this pull request?
    
    The test added in https://github.com/apache/spark/pull/44095 could be flaky 
because `MEDIUM_RESULTS_QUERY` could very quickly finish before interrupt was 
sent. Replace it with a query that sleeps 30 seconds, so that we are sure that 
interrupt runs before it finishes.
    
    ### Why are the changes needed?
    
    Remove test flakiness.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Rerun ReattachableExecuteSuite 100+ times to check it isn't flaky.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Github Copilot was assisting in some boilerplate auto-completion.
    
    Generated-by: Github Copilot
    
    Closes #44189 from juliuszsompolski/SPARK-46186-followup.
    
    Authored-by: Juliusz Sompolski <ju...@databricks.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../sql/connect/execution/ReattachableExecuteSuite.scala     | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git 
a/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
 
b/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
index 02b75f04495..f80229c6198 100644
--- 
a/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
+++ 
b/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
@@ -298,6 +298,15 @@ class ReattachableExecuteSuite extends 
SparkConnectServerTest {
   }
 
   test("SPARK-46186 interrupt directly after query start") {
+    // register a sleep udf in the session
+    val serverSession =
+      SparkConnectService.getOrCreateIsolatedSession(defaultUserId, 
defaultSessionId).session
+    serverSession.udf.register(
+      "sleep",
+      ((ms: Int) => {
+        Thread.sleep(ms);
+        ms
+      }))
     // This test depends on fast timing.
     // If something is wrong, it can fail only from time to time.
     withRawBlockingStub { stub =>
@@ -309,12 +318,13 @@ class ReattachableExecuteSuite extends 
SparkConnectServerTest {
         .setOperationId(operationId)
         .build()
       val iter = stub.executePlan(
-        buildExecutePlanRequest(buildPlan(MEDIUM_RESULTS_QUERY), operationId = 
operationId))
+        buildExecutePlanRequest(buildPlan("select sleep(30000) as s"), 
operationId = operationId))
       // wait for execute holder to exist, but the execute thread may not have 
started yet.
       Eventually.eventually(timeout(eventuallyTimeout)) {
         assert(SparkConnectService.executionManager.listExecuteHolders.length 
== 1)
       }
       stub.interrupt(interruptRequest)
+      // make sure the client gets the OPERATION_CANCELED error
       val e = intercept[StatusRuntimeException] {
         while (iter.hasNext) iter.next()
       }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org