This is an automated email from the ASF dual-hosted git repository.
HyukjinKwon pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 9692ec2 [SPARK-59137][PYTHON][TESTS] Update offline tests for the
reference __module__ on catalog classes and PythonEvalType
9692ec2 is described below
commit 9692ec21ad518ae546721e1894c5909a9583d941
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Tue Sep 1 07:35:10 2026 +0900
[SPARK-59137][PYTHON][TESTS] Update offline tests for the reference
__module__ on catalog classes and PythonEvalType
### What changes were proposed in this pull request?
Follow-up of #89 (SPARK-59117), which set an explicit `__module__` on the
PyO3
classes to the reference pyspark path. Two older offline tests in
`python/tests/test_dropin_offline.py` still asserted the pre-59117
`__module__ == "builtins"` and now fail in CI:
- `test_catalog_result_classes`: `CatalogMetadata` / `Database` / `Table` /
`Function` / `TablePartition` now report `pyspark.sql.catalog`.
- `test_python_eval_type_constants`: `PythonEvalType` now reports
`pyspark.util`.
This updates both to assert the reference module paths.
### Why are the changes needed?
The coverage job fails on `master` (and therefore on every PR branched from
it — e.g.
#90) at these two assertions. The expected values were verified against the
`#[pyclass(module = "...")]` declarations in `crates/pyspark-rs/src`
(`catalog.rs`,
`eval_type.rs`), and are consistent with the tests SPARK-59117 itself added
(`test_class_module_matches_reference_path` and the
`test_no_pyclass_reports_builtins_module`
guard, both of which pass).
### Does this PR introduce _any_ user-facing change?
No. Test-only change; no source behavior change.
### How was this patch tested?
`python -m py_compile` on the test file. The corrected expected values
match both the
Rust `module = "..."` declarations and the failing CI assertions
(`'pyspark.sql.catalog' == 'builtins'`, `'pyspark.util' == 'builtins'`).
Full suite
runs in CI.
This pull request and its description were written by Isaac.
Closes #91 from HyukjinKwon/fix-module-tests-59117.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 464ce0cde56e65689570d53476f47c1559b2f37c)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/tests/test_dropin_offline.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/python/tests/test_dropin_offline.py
b/python/tests/test_dropin_offline.py
index 07e2e66..8b78005 100644
--- a/python/tests/test_dropin_offline.py
+++ b/python/tests/test_dropin_offline.py
@@ -507,9 +507,10 @@ def test_catalog_result_classes():
Catalog, CatalogMetadata, Database, Table, Column, Function,
TablePartition,
)
import pyspark._pyspark as p
- # Rust-backed (defined in the extension module).
+ # Rust-backed, but reporting the reference `pyspark.sql.catalog` module
path
+ # (SPARK-59117), not "builtins".
for cls in (CatalogMetadata, Database, Table, Function, TablePartition):
- assert cls.__module__ == "builtins", f"{cls.__name__} should be
Rust-backed"
+ assert cls.__module__ == "pyspark.sql.catalog", f"{cls.__name__}
module = {cls.__module__}"
assert Column is p.CatalogColumn
@@ -557,7 +558,7 @@ def test_python_eval_type_constants():
assert PythonEvalType.SQL_SCALAR_ARROW_UDF == 250
assert PythonEvalType.SQL_TABLE_UDF == 300
assert PythonEvalType.SQL_ARROW_TABLE_UDF == 301
- assert PythonEvalType.__module__ == "builtins" # Rust-backed
+ assert PythonEvalType.__module__ == "pyspark.util" # reference path
(SPARK-59117)
def test_reexport_paths():
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
