This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new aaa2e5f6d95b [SPARK-60877][4.0][CORE][TEST] Add `gridTest` and
`namedGridTest` to `SparkFunSuite`
aaa2e5f6d95b is described below
commit aaa2e5f6d95bfbbfe7262766d2da482547563c54
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Nov 4 07:30:57 2026 -0800
[SPARK-60877][4.0][CORE][TEST] Add `gridTest` and `namedGridTest` to
`SparkFunSuite`
### What changes were proposed in this pull request?
This PR aims to add `gridTest` and `namedGridTest` to `SparkFunSuite` in
`branch-4.0`.
The two methods are the same code as `branch-4.1`, where SPARK-52405
(#51098) and SPARK-53522 (#52266) added them. `SparkFunSuite` has had them
since Apache Spark
[v4.1.0](https://github.com/apache/spark/releases/tag/v4.1.0) (2025-12-11), so
`branch-4.0` is the only `4.x` release branch without them.
```scala
protected def gridTest[A](testNamePrefix: String, testTags: Tag*)(params:
Seq[A])(
testFun: A => Unit): Unit = {
for (param <- params) {
test(testNamePrefix + s" ($param)", testTags: _*)(testFun(param))
}
}
protected def namedGridTest[A](testNamePrefix: String, testTags:
Tag*)(params: Map[String, A])(
testFun: A => Unit): Unit = {
for (param <- params) {
test(testNamePrefix + s" ${param._1}", testTags: _*)(testFun(param._2))
}
}
```
This PR also removes the `gridTest` of `SparkConnectServiceSuite` and its
unused `org.scalatest.Tag` import, as SPARK-52405 did. Otherwise, it fails to
compile because it needs the `override` modifier.
### Why are the changes needed?
A backport that uses `gridTest` is cherry-picked to `branch-4.0` without
conflicts, but it fails to compile there. This broke the `branch-4.0` test
compilation twice, and each time it needed a separate PR that replaced
`gridTest` with `Seq.foreach`.
- SPARK-60430 (#59680): 682e33b84d5 (2026-10-22) added `gridTest` to
`RecoverySuite`. Fixed by #59698.
```
[error]
.../core/src/test/scala/org/apache/spark/deploy/master/RecoverySuite.scala:516:3:
not found: value gridTest
```
- SPARK-60835 (#60050): 1ff8a134385 (2026-11-04) added `gridTest` to
`LabelPropagationSuite`. Fixed by #60094.
```
[error]
.../graphx/src/test/scala/org/apache/spark/graphx/lib/LabelPropagationSuite.scala:45:3:
not found: value gridTest
[error]
.../graphx/src/test/scala/org/apache/spark/graphx/lib/LabelPropagationSuite.scala:48:32:
type mismatch;
[error] found : Any
[error] required: org.apache.spark.graphx.VertexId
```
`gridTest` is used more and more in `master` (57 test files as of today),
so more backports are likely to break the same way. With this PR, such
backports compile on `branch-4.0` as they are.
The existing `Seq.foreach` code in `RecoverySuite` and
`LabelPropagationSuite` is kept because it generates the same test names as
`gridTest`.
### Does this PR introduce _any_ user-facing change?
No, this is a test-only change.
### How was this patch tested?
Pass the CIs. I also ran the following manually on `branch-4.0` with JDK 17.
```
$ build/sbt core/Test/compile graphx/Test/compile connect/Test/compile \
core/Test/scalastyle graphx/Test/scalastyle connect/Test/scalastyle \
'graphx/testOnly *LabelPropagationSuite' \
'connect/testOnly *SparkConnectServiceSuite -- -z SPARK-43923'
...
[info] scalastyle Found 0 errors
...
[info] scalastyle Found 0 errors
...
[info] scalastyle Found 0 errors
...
[info] Tests: succeeded 5, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
...
[info] Tests: succeeded 12, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```
To check that this PR prevents the breakage, I also applied the
`SparkFunSuite` change to 1ff8a134385, before #60094, and ran
`LabelPropagationSuite` with its original `gridTest` call. It compiled and
passed with the same test names.
```
[info] - Label propagation breaks label ties with the lowest vertex ID
((1,2)) (121 milliseconds)
[info] - Label propagation breaks label ties with the lowest vertex ID
((-2,-1)) (109 milliseconds)
[info] - Label propagation breaks label ties with the lowest vertex ID
((-9223372036854775808,9223372036854775807)) (107 milliseconds)
...
[info] Tests: succeeded 5, failed 0, canceled 0, ignored 0, pending 0
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #60095 from dongjoon-hyun/d/gridtest-backport-branch-4-0-6e694d.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
core/src/test/scala/org/apache/spark/SparkFunSuite.scala | 14 ++++++++++++++
.../sql/connect/planner/SparkConnectServiceSuite.scala | 8 --------
2 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
b/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
index e38efc27b78f..186d02f447af 100644
--- a/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
+++ b/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
@@ -236,6 +236,20 @@ abstract class SparkFunSuite
}
}
+ protected def gridTest[A](testNamePrefix: String, testTags: Tag*)(params:
Seq[A])(
+ testFun: A => Unit): Unit = {
+ for (param <- params) {
+ test(testNamePrefix + s" ($param)", testTags: _*)(testFun(param))
+ }
+ }
+
+ protected def namedGridTest[A](testNamePrefix: String, testTags:
Tag*)(params: Map[String, A])(
+ testFun: A => Unit): Unit = {
+ for (param <- params) {
+ test(testNamePrefix + s" ${param._1}", testTags: _*)(testFun(param._2))
+ }
+ }
+
/**
* Creates a temporary directory, which is then passed to `f` and will be
deleted after `f`
* returns.
diff --git
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
index c6daa92e9735..5d04d8409542 100644
---
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
+++
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
@@ -30,7 +30,6 @@ import org.apache.arrow.memory.RootAllocator
import org.apache.arrow.vector.{BigIntVector, Float8Vector}
import org.apache.arrow.vector.ipc.ArrowStreamReader
import org.mockito.Mockito.when
-import org.scalatest.Tag
import org.scalatest.concurrent.Eventually
import org.scalatest.time.SpanSugar.convertIntToGrainOfTime
import org.scalatestplus.mockito.MockitoSugar
@@ -864,13 +863,6 @@ class SparkConnectServiceSuite
}
}
- protected def gridTest[A](testNamePrefix: String, testTags: Tag*)(params:
Seq[A])(
- testFun: A => Unit): Unit = {
- for (param <- params) {
- test(testNamePrefix + s" ($param)", testTags: _*)(testFun(param))
- }
- }
-
class VerifyEvents(val sparkContext: SparkContext) {
val listener: MockSparkListener = new MockSparkListener()
val listenerBus = sparkContext.listenerBus
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]