This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7e7380ff590b [SPARK-53424][PYTHON][TESTS] Hide traceback in
`assertSchemaEqual/assertDataFrameEqual/assert_rows_equal`
7e7380ff590b is described below
commit 7e7380ff590bcdf498fb49cd623857415be08c10
Author: Allison Wang <allison.w...@databricks.com>
AuthorDate: Sat Aug 30 13:14:22 2025 -0700
[SPARK-53424][PYTHON][TESTS] Hide traceback in
`assertSchemaEqual/assertDataFrameEqual/assert_rows_equal`
### What changes were proposed in this pull request?
This PR hides traceback for several utility functions in testing utils.
### Why are the changes needed?
To hide unnecessary information when using testing utils like
assertDataFrameEqual. The error message is too verbose. For example we do not
need to show this `assert_rows_equal` function body:
```
python/pyspark/testing/utils.py:1114: in assertDataFrameEqual
assert_rows_equal(actual_list, expected_list, maxErrors=maxErrors,
showOnlyDiff=showOnlyDiff)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def assert_rows_equal(
rows1: List[Row], rows2: List[Row], maxErrors: int = None,
showOnlyDiff: bool = False
):
zipped = list(zip_longest(rows1, rows2))
diff_rows_cnt = 0
diff_rows = []
has_diff_rows = False
rows_str1 = ""
rows_str2 = ""
# count different rows
for r1, r2 in zipped:
if not compare_rows(r1, r2):
...
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual testing
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #52129 from allisonwang-db/minor-traceback.
Authored-by: Allison Wang <allison.w...@databricks.com>
Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
python/pyspark/testing/utils.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/python/pyspark/testing/utils.py b/python/pyspark/testing/utils.py
index e1d90102a374..2c13d2f74026 100644
--- a/python/pyspark/testing/utils.py
+++ b/python/pyspark/testing/utils.py
@@ -459,6 +459,7 @@ def assertSchemaEqual(
ignoreColumnOrder: bool = False,
ignoreColumnName: bool = False,
):
+ __tracebackhide__ = True
r"""
A util function to assert equality between DataFrame schemas `actual` and
`expected`.
@@ -648,6 +649,7 @@ def assertDataFrameEqual(
showOnlyDiff: bool = False,
includeDiffRows=False,
):
+ __tracebackhide__ = True
r"""
A util function to assert equality between `actual` and `expected`
(DataFrames or lists of Rows), with optional parameters `checkRowOrder`,
`rtol`, and `atol`.
@@ -1034,6 +1036,7 @@ def assertDataFrameEqual(
def assert_rows_equal(
rows1: List[Row], rows2: List[Row], maxErrors: int = None,
showOnlyDiff: bool = False
):
+ __tracebackhide__ = True
zipped = list(zip_longest(rows1, rows2))
diff_rows_cnt = 0
diff_rows = []
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
