This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.0 by this push:
     new aaa2e5f6d95b [SPARK-60877][4.0][CORE][TEST] Add `gridTest` and 
`namedGridTest` to `SparkFunSuite`
aaa2e5f6d95b is described below

commit aaa2e5f6d95bfbbfe7262766d2da482547563c54
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Nov 4 07:30:57 2026 -0800

    [SPARK-60877][4.0][CORE][TEST] Add `gridTest` and `namedGridTest` to 
`SparkFunSuite`
    
    ### What changes were proposed in this pull request?
    
    This PR aims to add `gridTest` and `namedGridTest` to `SparkFunSuite` in 
`branch-4.0`.
    
    The two methods are the same code as `branch-4.1`, where SPARK-52405 
(#51098) and SPARK-53522 (#52266) added them. `SparkFunSuite` has had them 
since Apache Spark 
[v4.1.0](https://github.com/apache/spark/releases/tag/v4.1.0) (2025-12-11), so 
`branch-4.0` is the only `4.x` release branch without them.
    
    ```scala
    protected def gridTest[A](testNamePrefix: String, testTags: Tag*)(params: 
Seq[A])(
      testFun: A => Unit): Unit = {
      for (param <- params) {
        test(testNamePrefix + s" ($param)", testTags: _*)(testFun(param))
      }
    }
    
    protected def namedGridTest[A](testNamePrefix: String, testTags: 
Tag*)(params: Map[String, A])(
      testFun: A => Unit): Unit = {
      for (param <- params) {
        test(testNamePrefix + s" ${param._1}", testTags: _*)(testFun(param._2))
      }
    }
    ```
    
    This PR also removes the `gridTest` of `SparkConnectServiceSuite` and its 
unused `org.scalatest.Tag` import, as SPARK-52405 did. Otherwise, it fails to 
compile because it needs the `override` modifier.
    
    ### Why are the changes needed?
    
    A backport that uses `gridTest` is cherry-picked to `branch-4.0` without 
conflicts, but it fails to compile there. This broke the `branch-4.0` test 
compilation twice, and each time it needed a separate PR that replaced 
`gridTest` with `Seq.foreach`.
    
    - SPARK-60430 (#59680): 682e33b84d5 (2026-10-22) added `gridTest` to 
`RecoverySuite`. Fixed by #59698.
      ```
      [error] 
.../core/src/test/scala/org/apache/spark/deploy/master/RecoverySuite.scala:516:3:
 not found: value gridTest
      ```
    - SPARK-60835 (#60050): 1ff8a134385 (2026-11-04) added `gridTest` to 
`LabelPropagationSuite`. Fixed by #60094.
      ```
      [error] 
.../graphx/src/test/scala/org/apache/spark/graphx/lib/LabelPropagationSuite.scala:45:3:
 not found: value gridTest
      [error] 
.../graphx/src/test/scala/org/apache/spark/graphx/lib/LabelPropagationSuite.scala:48:32:
 type mismatch;
      [error]  found   : Any
      [error]  required: org.apache.spark.graphx.VertexId
      ```
    
    `gridTest` is used more and more in `master` (57 test files as of today), 
so more backports are likely to break the same way. With this PR, such 
backports compile on `branch-4.0` as they are.
    
    The existing `Seq.foreach` code in `RecoverySuite` and 
`LabelPropagationSuite` is kept because it generates the same test names as 
`gridTest`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, this is a test-only change.
    
    ### How was this patch tested?
    
    Pass the CIs. I also ran the following manually on `branch-4.0` with JDK 17.
    
    ```
    $ build/sbt core/Test/compile graphx/Test/compile connect/Test/compile \
        core/Test/scalastyle graphx/Test/scalastyle connect/Test/scalastyle \
        'graphx/testOnly *LabelPropagationSuite' \
        'connect/testOnly *SparkConnectServiceSuite -- -z SPARK-43923'
    ...
    [info] scalastyle Found 0 errors
    ...
    [info] scalastyle Found 0 errors
    ...
    [info] scalastyle Found 0 errors
    ...
    [info] Tests: succeeded 5, failed 0, canceled 0, ignored 0, pending 0
    [info] All tests passed.
    ...
    [info] Tests: succeeded 12, failed 0, canceled 0, ignored 0, pending 0
    [info] All tests passed.
    ```
    
    To check that this PR prevents the breakage, I also applied the 
`SparkFunSuite` change to 1ff8a134385, before #60094, and ran 
`LabelPropagationSuite` with its original `gridTest` call. It compiled and 
passed with the same test names.
    
    ```
    [info] - Label propagation breaks label ties with the lowest vertex ID 
((1,2)) (121 milliseconds)
    [info] - Label propagation breaks label ties with the lowest vertex ID 
((-2,-1)) (109 milliseconds)
    [info] - Label propagation breaks label ties with the lowest vertex ID 
((-9223372036854775808,9223372036854775807)) (107 milliseconds)
    ...
    [info] Tests: succeeded 5, failed 0, canceled 0, ignored 0, pending 0
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    Closes #60095 from dongjoon-hyun/d/gridtest-backport-branch-4-0-6e694d.
    
    Authored-by: Dongjoon Hyun <[email protected]>
    Signed-off-by: Dongjoon Hyun <[email protected]>
---
 core/src/test/scala/org/apache/spark/SparkFunSuite.scala   | 14 ++++++++++++++
 .../sql/connect/planner/SparkConnectServiceSuite.scala     |  8 --------
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/core/src/test/scala/org/apache/spark/SparkFunSuite.scala 
b/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
index e38efc27b78f..186d02f447af 100644
--- a/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
+++ b/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
@@ -236,6 +236,20 @@ abstract class SparkFunSuite
     }
   }
 
+  protected def gridTest[A](testNamePrefix: String, testTags: Tag*)(params: 
Seq[A])(
+    testFun: A => Unit): Unit = {
+    for (param <- params) {
+      test(testNamePrefix + s" ($param)", testTags: _*)(testFun(param))
+    }
+  }
+
+  protected def namedGridTest[A](testNamePrefix: String, testTags: 
Tag*)(params: Map[String, A])(
+    testFun: A => Unit): Unit = {
+    for (param <- params) {
+      test(testNamePrefix + s" ${param._1}", testTags: _*)(testFun(param._2))
+    }
+  }
+
   /**
    * Creates a temporary directory, which is then passed to `f` and will be 
deleted after `f`
    * returns.
diff --git 
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
 
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
index c6daa92e9735..5d04d8409542 100644
--- 
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
+++ 
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
@@ -30,7 +30,6 @@ import org.apache.arrow.memory.RootAllocator
 import org.apache.arrow.vector.{BigIntVector, Float8Vector}
 import org.apache.arrow.vector.ipc.ArrowStreamReader
 import org.mockito.Mockito.when
-import org.scalatest.Tag
 import org.scalatest.concurrent.Eventually
 import org.scalatest.time.SpanSugar.convertIntToGrainOfTime
 import org.scalatestplus.mockito.MockitoSugar
@@ -864,13 +863,6 @@ class SparkConnectServiceSuite
     }
   }
 
-  protected def gridTest[A](testNamePrefix: String, testTags: Tag*)(params: 
Seq[A])(
-      testFun: A => Unit): Unit = {
-    for (param <- params) {
-      test(testNamePrefix + s" ($param)", testTags: _*)(testFun(param))
-    }
-  }
-
   class VerifyEvents(val sparkContext: SparkContext) {
     val listener: MockSparkListener = new MockSparkListener()
     val listenerBus = sparkContext.listenerBus


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to