This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 982c7268f4c [SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness
of `ReattachableExecuteSuite`
982c7268f4c is described below
commit 982c7268f4c7ea1fa03ea679146f9e83f31bece7
Author: Juliusz Sompolski <[email protected]>
AuthorDate: Wed Dec 6 10:57:45 2023 -0800
[SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness of
`ReattachableExecuteSuite`
### What changes were proposed in this pull request?
The test added in https://github.com/apache/spark/pull/44095 could be flaky
because `MEDIUM_RESULTS_QUERY` could very quickly finish before interrupt was
sent. Replace it with a query that sleeps 30 seconds, so that we are sure that
interrupt runs before it finishes.
### Why are the changes needed?
Remove test flakiness.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Rerun ReattachableExecuteSuite 100+ times to check it isn't flaky.
### Was this patch authored or co-authored using generative AI tooling?
Github Copilot was assisting in some boilerplate auto-completion.
Generated-by: Github Copilot
Closes #44189 from juliuszsompolski/SPARK-46186-followup.
Authored-by: Juliusz Sompolski <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../sql/connect/execution/ReattachableExecuteSuite.scala | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git
a/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
b/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
index 02b75f04495..f80229c6198 100644
---
a/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
+++
b/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala
@@ -298,6 +298,15 @@ class ReattachableExecuteSuite extends
SparkConnectServerTest {
}
test("SPARK-46186 interrupt directly after query start") {
+ // register a sleep udf in the session
+ val serverSession =
+ SparkConnectService.getOrCreateIsolatedSession(defaultUserId,
defaultSessionId).session
+ serverSession.udf.register(
+ "sleep",
+ ((ms: Int) => {
+ Thread.sleep(ms);
+ ms
+ }))
// This test depends on fast timing.
// If something is wrong, it can fail only from time to time.
withRawBlockingStub { stub =>
@@ -309,12 +318,13 @@ class ReattachableExecuteSuite extends
SparkConnectServerTest {
.setOperationId(operationId)
.build()
val iter = stub.executePlan(
- buildExecutePlanRequest(buildPlan(MEDIUM_RESULTS_QUERY), operationId =
operationId))
+ buildExecutePlanRequest(buildPlan("select sleep(30000) as s"),
operationId = operationId))
// wait for execute holder to exist, but the execute thread may not have
started yet.
Eventually.eventually(timeout(eventuallyTimeout)) {
assert(SparkConnectService.executionManager.listExecuteHolders.length
== 1)
}
stub.interrupt(interruptRequest)
+ // make sure the client gets the OPERATION_CANCELED error
val e = intercept[StatusRuntimeException] {
while (iter.hasNext) iter.next()
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
