This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new e3ae7a0ff604 [SPARK-54551][CONNECT][TESTS] Fix random hangs in
SparkConnectJdbcDataTypeSuite
e3ae7a0ff604 is described below
commit e3ae7a0ff6048f62c97d5695c1df12d964a9d70b
Author: vinodkc <[email protected]>
AuthorDate: Sat Nov 29 12:58:08 2025 -0800
[SPARK-54551][CONNECT][TESTS] Fix random hangs in
SparkConnectJdbcDataTypeSuite
### What changes were proposed in this pull request?
Added explicit stdout/stderr redirection to Redirect.DISCARD for the child
Spark Connect server process in non-debug mode in SparkConnectServerUtils.
### Why are the changes needed?
SparkConnectJdbcDataTypeSuite randomly hangs because the child server
process blocks on write() calls when stdout/stderr pipe buffers fill up.
Without consuming the output, the buffers reach capacity and cause the process
to block indefinitely.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually tested verified `SparkConnectJdbcDataTypeSuite` no longer randomly
hangs and all tests pass consistently.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53261 from vinodkc/br_handle_std_pipe.
Authored-by: vinodkc <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 99270f0b7224eec448f83eb74f24fdbfefba26b4)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala | 5 +++++
1 file changed, 5 insertions(+)
diff --git
a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala
b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala
index efac3bc7561f..6229e6e299db 100644
---
a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala
+++
b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala
@@ -93,6 +93,11 @@ object SparkConnectServerUtils {
if (isDebug) {
builder.redirectError(Redirect.INHERIT)
builder.redirectOutput(Redirect.INHERIT)
+ } else {
+ // If output is not consumed, the stdout/stderr pipe buffers will fill
up,
+ // causing the server process to block on write() calls
+ builder.redirectError(Redirect.DISCARD)
+ builder.redirectOutput(Redirect.DISCARD)
}
val process = builder.start()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
