This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b154623d9fac [MINOR][TESTS] Always remove spark.master in
ReusedConnectTestCase
b154623d9fac is described below
commit b154623d9fac162d227e1f779b962ec37d0cbb7d
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Thu Jun 27 12:30:38 2024 +0900
[MINOR][TESTS] Always remove spark.master in ReusedConnectTestCase
### What changes were proposed in this pull request?
This PR proposes to always remove `spark.master` in `ReusedConnectTestCase`.
### Why are the changes needed?
More just for sanity. The tests should only run with `spark.remote`
### Does this PR introduce _any_ user-facing change?
No, test-only
### How was this patch tested?
Fixed unittest
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #47113 from HyukjinKwon/minor-test-test.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/testing/connectutils.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/python/pyspark/testing/connectutils.py
b/python/pyspark/testing/connectutils.py
index b3004693724b..2f18cd8a6ccd 100644
--- a/python/pyspark/testing/connectutils.py
+++ b/python/pyspark/testing/connectutils.py
@@ -177,6 +177,8 @@ class ReusedConnectTestCase(unittest.TestCase,
SQLTestUtils, PySparkErrorTestUti
conf = SparkConf(loadDefaults=False)
# Make the server terminate reattachable streams every 1 second and
123 bytes,
# to make the tests exercise reattach.
+ if conf._jconf is not None:
+ conf._jconf.remove("spark.master")
conf.set("spark.connect.execute.reattachable.senderMaxStreamDuration",
"1s")
conf.set("spark.connect.execute.reattachable.senderMaxStreamSize",
"123")
return conf
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
