This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b284a2c50e96 [SPARK-54101][GEO][FOLLOWUP] Re-enable Scala/Python
parity check for ST functions
b284a2c50e96 is described below
commit b284a2c50e9614979322d6a70dd8c346fdf507d8
Author: Uros Bojanic <[email protected]>
AuthorDate: Wed Nov 5 21:59:58 2025 -0800
[SPARK-54101][GEO][FOLLOWUP] Re-enable Scala/Python parity check for ST
functions
### What changes were proposed in this pull request?
Re-enable Scala/Python parity check for ST geospatial functions in
`test_function_parity`.
### Why are the changes needed?
The test was temporarily disabled in
https://github.com/apache/spark/pull/52803, but the corresponding functions
have been subsequently added on PySpark side as part of
https://github.com/apache/spark/pull/52849.
### Does this PR introduce _any_ user-facing change?
Yes, casting `GEOGRAPHY(<srid>)` to `GEOGRAPHY(ANY)` is now allowed.
### How was this patch tested?
Existing tests suffice:
- `test_functions`
Closes #52907 from uros-db/geo-function_parity.
Authored-by: Uros Bojanic <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/pyspark/sql/tests/test_functions.py | 4 ----
1 file changed, 4 deletions(-)
diff --git a/python/pyspark/sql/tests/test_functions.py
b/python/pyspark/sql/tests/test_functions.py
index 07c0c9aaf524..0f646b3da549 100644
--- a/python/pyspark/sql/tests/test_functions.py
+++ b/python/pyspark/sql/tests/test_functions.py
@@ -80,10 +80,6 @@ class FunctionsTestsMixin:
missing_in_py = jvm_fn_set.difference(py_fn_set)
- # Temporarily disable Scala/Python parity check for ST geospatial
functions, while the
- # feature is under development. Once the geospatial module is stable,
remove this.
- missing_in_py = {fn for fn in missing_in_py if not
fn.startswith("st_")}
-
# Functions that we expect to be missing in python until they are
added to pyspark
expected_missing_in_py = set()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
