This is an automated email from the ASF dual-hosted git repository.
sarutak pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 967804e1bfe5 [SPARK-57348][PYTHON][TESTS][4.0] Replace sql_keywords
doctest show() with columns check
967804e1bfe5 is described below
commit 967804e1bfe515809e7d1e9f8faf8e814ec41a9c
Author: Kousuke Saruta <[email protected]>
AuthorDate: Fri Jun 12 04:54:13 2026 +0900
[SPARK-57348][PYTHON][TESTS][4.0] Replace sql_keywords doctest show() with
columns check
### What changes were proposed in this pull request?
This PR backports #56406 to `branch-4.0`.
Replace the `show()`-based doctest for `sql_keywords()` with a `.columns`
check.
### Why are the changes needed?
SPARK-57133 (#56247) added 7 new non-reserved keywords (BIN, WIDTH, ALIGN,
etc.), which changed the top-20 row output of `sql_keywords().show()` and
consequently the column width in the formatted output. This broke the
`pyspark-connect-old-client` CI job, which runs `branch-4.0` client tests
against a `master` server. The `branch-4.0` doctest still expects the old
column width.
https://github.com/sarutak/spark/actions/runs/27188469096/job/80265105548
```
**********************************************************************
File "/__w/spark/spark-4.0/python/pyspark/sql/connect/tvf.py", line ?, in
pyspark.sql.connect.tvf.TableValuedFunction.sql_keywords
Failed example:
spark.tvf.sql_keywords().show()
Expected:
+-------------+--------+
| keyword|reserved|
+-------------+--------+
...
+-------------+--------+...
Got:
+----------+--------+
| keyword|reserved|
+----------+--------+
| ADD| false|
| AFTER| false|
| AGGREGATE| false|
| ALIGN| false|
| ALL| false|
| ALTER| false|
| ALWAYS| false|
| ANALYZE| false|
| AND| false|
| ANTI| false|
| ANY| false|
| ANY_VALUE| false|
| APPROX| false|
| ARCHIVE| false|
| ARRAY| false|
| AS| false|
| ASC| false|
|ASENSITIVE| false|
| AT| false|
| ATOMIC| false|
+----------+--------+
only showing top 20 rows
**********************************************************************
1 of 1 in pyspark.sql.connect.tvf.TableValuedFunction.sql_keywords
***Test Failed*** 1 failures.
```
The `show()` output is inherently fragile for this TVF because any keyword
addition changes the formatting. Since a dedicated unittest
(`test_sql_keywords` in `test_tvf.py`) already verifies the full output via
`assertDataFrameEqual`, the doctest only needs to confirm that the method
returns a valid DataFrame. Using `.columns` achieves this without being
sensitive to keyword list changes.
### Does this PR introduce *any* user-facing change?
No.
### How was this patch tested?
Existing `test_sql_keywords` unittest continues to pass.
### Was this patch authored or co-authored using generative AI tooling?
Kiro CLI / Claude
Closes #56452 from sarutak/fix-sql-keywords-doctest-4.0.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Kousuke Saruta <[email protected]>
---
python/pyspark/sql/tvf.py | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/python/pyspark/sql/tvf.py b/python/pyspark/sql/tvf.py
index b34877b03311..2aa6073508b7 100644
--- a/python/pyspark/sql/tvf.py
+++ b/python/pyspark/sql/tvf.py
@@ -550,12 +550,8 @@ class TableValuedFunction:
Examples
--------
- >>> spark.tvf.sql_keywords().show()
- +-------------+--------+
- | keyword|reserved|
- +-------------+--------+
- ...
- +-------------+--------+...
+ >>> spark.tvf.sql_keywords().columns
+ ['keyword', 'reserved']
"""
return self._fn("sql_keywords")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
