This is an automated email from the ASF dual-hosted git repository.
hvanhovell pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 71970798a5bd [SPARK-51192][CONNECT] Expose
`processWithoutResponseObserverForTesting` in `SparkConnectPlanner`
71970798a5bd is described below
commit 71970798a5bd4a6c2908cad2b084556a798f2d66
Author: vicennial <[email protected]>
AuthorDate: Mon Feb 17 09:41:52 2025 -0400
[SPARK-51192][CONNECT] Expose `processWithoutResponseObserverForTesting` in
`SparkConnectPlanner`
### What changes were proposed in this pull request?
Adds a new `processWithoutResponseObserverForTesting` private-package
method scoped to `planner`. This method is called by
`SparkConnectPlannerTestUtils#transform`.
### Why are the changes needed?
https://github.com/apache/spark/pull/47816 attempted to move `MockObserver`
into source code to address compilation errors when open-source libraries
attempt to test their command plugin extensions via the
`SparkConnectPlannerUtils`.
However, this isn't enough as the error `java.lang.NoSuchMethodError: 'void
org.apache.spark.sql.connect.planner.SparkConnectPlanner.process(org.apache.spark.connect.proto.Command,
io.grpc.stub.StreamObserver` continues to be seen.
To address this shading issue, we move the creation of the `MockObserver`
to the source code.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #49921 from vicennial/SPARK-51192.
Authored-by: vicennial <[email protected]>
Signed-off-by: Herman van Hovell <[email protected]>
(cherry picked from commit 479e0b34c4c99e6bd5ebf8ce1524d8616e6e3d3e)
Signed-off-by: Herman van Hovell <[email protected]>
---
.../apache/spark/sql/connect/planner/SparkConnectPlanner.scala | 9 +++++++++
.../spark/sql/connect/planner/SparkConnectPlannerTestUtils.scala | 2 +-
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala
index f8fd56f38f9b..8aaa3a7151a2 100644
---
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala
+++
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala
@@ -2474,6 +2474,15 @@ class SparkConnectPlanner(
}
}
+ /**
+ * Exposed for testing. Processes a command without a response observer.
+ *
+ * Called only from SparkConnectPlannerTestUtils.
+ */
+ private[planner] def processWithoutResponseObserverForTesting(command:
proto.Command): Unit = {
+ process(command, new MockObserver())
+ }
+
def process(
command: proto.Command,
responseObserver: StreamObserver[ExecutePlanResponse]): Unit = {
diff --git
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectPlannerTestUtils.scala
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectPlannerTestUtils.scala
index ee830c3b9672..9f6278771a8b 100644
---
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectPlannerTestUtils.scala
+++
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectPlannerTestUtils.scala
@@ -31,7 +31,7 @@ object SparkConnectPlannerTestUtils {
def transform(spark: SparkSession, command: proto.Command): Unit = {
val executeHolder = buildExecutePlanHolder(spark, command)
- new SparkConnectPlanner(executeHolder).process(command, new MockObserver())
+ new
SparkConnectPlanner(executeHolder).processWithoutResponseObserverForTesting(command)
}
private def buildExecutePlanHolder(
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
