This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new d4703f298c99 [SPARK-52420][PYTHON][TESTS] Make
test_udtf_with_invalid_return_type compatible with Python only client
d4703f298c99 is described below
commit d4703f298c99f61a2eed23339acd821f75c7afcf
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Mon Jun 9 03:09:42 2025 -0700
[SPARK-52420][PYTHON][TESTS] Make test_udtf_with_invalid_return_type
compatible with Python only client
This PR is a followup of https://github.com/apache/spark/pull/50997 that
backports the test changes into branch-3.5.
To make the build pass
(https://github.com/apache/spark/actions/runs/15522509263/job/43697654408).
```
======================================================================
FAIL [0.015s]: test_udtf_with_invalid_return_type
(pyspark.sql.tests.connect.test_parity_udtf.UDTFParityTests.test_udtf_with_invalid_return_type)
----------------------------------------------------------------------
pyspark.errors.exceptions.connect.SparkConnectGrpcException:
(org.apache.spark.sql.connect.common.InvalidPlanInput)
[INVALID_SCHEMA_TYPE_NON_STRUCT] Invalid schema type. Expect a struct type, but
got "INT". SQLSTATE: 42K09
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/runner/work/spark/spark-3.5/python/pyspark/sql/tests/connect/test_parity_udtf.py",
line 57, in test_udtf_with_invalid_return_type
with self.assertRaisesRegex(
AssertionError: "Invalid Python user-defined table function return type."
does not match "(org.apache.spark.sql.connect.common.InvalidPlanInput)
[INVALID_SCHEMA_TYPE_NON_STRUCT] Invalid schema type. Expect a struct type, but
got "INT". SQLSTATE: 42K09"
----------------------------------------------------------------------
```
Now that the error message has changed, the tests in branch-3.5 fails.
No, test-only.
Manually.
No.
Closes #51121 from HyukjinKwon/SPARK-52420.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit c75e7c78b2d15b741698ce2c2f98e3c7e877ab91)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/tests/connect/test_parity_udtf.py | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/python/pyspark/sql/tests/connect/test_parity_udtf.py
b/python/pyspark/sql/tests/connect/test_parity_udtf.py
index 1d33e5cb6a52..b11483f4e8f4 100644
--- a/python/pyspark/sql/tests/connect/test_parity_udtf.py
+++ b/python/pyspark/sql/tests/connect/test_parity_udtf.py
@@ -56,9 +56,7 @@ class UDTFParityTests(BaseUDTFTestsMixin,
ReusedConnectTestCase):
def eval(self, a: int):
yield a + 1,
- with self.assertRaisesRegex(
- InvalidPlanInput, "Invalid Python user-defined table function
return type."
- ):
+ with self.assertRaisesRegex(InvalidPlanInput, "Invalid.*type"):
TestUDTF(lit(1)).collect()
@unittest.skip("Spark Connect does not support broadcast but the test
depends on it.")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
