This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 2e4dedf6171a
[SPARK-54481][SPARK-54495][SPARK-54483][TESTS][PYTHON][CONNECT] Reenable a few
spark connect tests
2e4dedf6171a is described below
commit 2e4dedf6171ad6a429ced2c33394801e1b09e4e7
Author: xianzhe-databricks <[email protected]>
AuthorDate: Mon Dec 1 18:47:36 2025 +0900
[SPARK-54481][SPARK-54495][SPARK-54483][TESTS][PYTHON][CONNECT] Reenable a
few spark connect tests
### What changes were proposed in this pull request?
* This PR re-enables 3 tests, which only fail during the cross-version
scenario: Spark Connect 4.0 client <-> 4.2 Spark Server
* We fixed these tests on the 4.0 branch separately in another PR
https://github.com/apache/spark/pull/53250
* ‼️ This PR needs to be backported to `branch-4.1`. How can I make it
possible? HyukjinKwon
### Why are the changes needed?
To regain test coverage
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #53235 from xianzhe-databricks/enable-tests.
Authored-by: xianzhe-databricks <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 54cde812c2657717651156636041a84556619a5b)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/pandas/tests/test_typedef.py | 3 ---
python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py | 4 ----
python/pyspark/sql/tests/pandas/test_pandas_map.py | 3 ---
3 files changed, 10 deletions(-)
diff --git a/python/pyspark/pandas/tests/test_typedef.py
b/python/pyspark/pandas/tests/test_typedef.py
index 29d1c170f065..54a03200c05c 100644
--- a/python/pyspark/pandas/tests/test_typedef.py
+++ b/python/pyspark/pandas/tests/test_typedef.py
@@ -311,9 +311,6 @@ class TypeHintTestsMixin:
self.assertRaisesRegex(TypeError, "object.*not understood",
try_infer_return_type)
- @unittest.skipIf(
- os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1",
"SPARK-54495: To be reenabled"
- )
def test_as_spark_type_pandas_on_spark_dtype(self):
type_mapper = {
# binary
diff --git a/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
b/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
index 0a77334ff85e..0d91da035497 100644
--- a/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
+++ b/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
@@ -18,7 +18,6 @@
import unittest
import logging
from typing import cast
-import os
from pyspark.sql import functions as sf
from pyspark.sql.functions import pandas_udf, udf
@@ -244,9 +243,6 @@ class CogroupedApplyInPandasTestsMixin:
self._test_merge_empty(fn=merge_pandas)
- @unittest.skipIf(
- os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1",
"SPARK-54481: To be reenabled"
- )
def test_apply_in_pandas_returning_incompatible_type(self):
with self.quiet():
self.check_apply_in_pandas_returning_incompatible_type()
diff --git a/python/pyspark/sql/tests/pandas/test_pandas_map.py
b/python/pyspark/sql/tests/pandas/test_pandas_map.py
index f816b09fe5ae..946d56f2fe63 100644
--- a/python/pyspark/sql/tests/pandas/test_pandas_map.py
+++ b/python/pyspark/sql/tests/pandas/test_pandas_map.py
@@ -273,9 +273,6 @@ class MapInPandasTestsMixin:
actual = df.repartition(1).mapInPandas(f, "id long, value
long").collect()
self.assertEqual(actual, expected)
- @unittest.skipIf(
- os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1",
"SPARK-54483: To be reenabled"
- )
def test_dataframes_with_incompatible_types(self):
with self.quiet():
self.check_dataframes_with_incompatible_types()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
