This is an automated email from the ASF dual-hosted git repository.
Yicong-Huang pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 81613116fa0c [SPARK-58179][PYTHON] Simplify namespace list
construction in PySpark Catalog
81613116fa0c is described below
commit 81613116fa0c2f51f0432a9298001312e9f59b95
Author: Matthew B. <[email protected]>
AuthorDate: Sun Jul 19 19:03:36 2026 +0000
[SPARK-58179][PYTHON] Simplify namespace list construction in PySpark
Catalog
### What changes were proposed in this pull request?
Replace the manual index-range comprehension used to turn a py4j
`JavaArray` into a Python list with a plain `list()` call, in
`python/pyspark/sql/catalog.py` (5 identical sites in `listViews`,
`listTables`, `getTable`, `listFunctions`, `getFunction`):
[SPARK-58179](https://issues.apache.org/jira/browse/SPARK-58179)
```diff
# before
- if jnamespace is not None:
- namespace = [jnamespace[i] for i in range(0, len(jnamespace))]
- else:
- namespace = None
# after
+ namespace = list(jnamespace) if jnamespace is not None else None
```
### Why are the changes needed?
Readability and de-duplication: five copies of a 4-line block collapse to
one idiomatic line each (5 insertions, 20 deletions). `list(jnamespace)` is the
standard way to materialize a sized, indexable py4j `JavaArray`.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing catalog tests (`pyspark.sql.tests.test_catalog`). No behavior
change, so no new tests added.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #57324 from Ma77Ball/SPARK-58179.
Authored-by: Matthew B. <[email protected]>
Signed-off-by: Yicong-Huang <[email protected]>
(cherry picked from commit 0c97b7d6d79aa56fe19d7073bdd0cd28932be355)
Signed-off-by: Yicong-Huang <[email protected]>
---
python/pyspark/sql/catalog.py | 25 +++++--------------------
1 file changed, 5 insertions(+), 20 deletions(-)
diff --git a/python/pyspark/sql/catalog.py b/python/pyspark/sql/catalog.py
index 10e4ec5fc927..e1a494fda91b 100644
--- a/python/pyspark/sql/catalog.py
+++ b/python/pyspark/sql/catalog.py
@@ -365,10 +365,7 @@ class Catalog:
while iter.hasNext():
jtable = iter.next()
jnamespace = jtable.namespace()
- if jnamespace is not None:
- namespace = [jnamespace[i] for i in range(0, len(jnamespace))]
- else:
- namespace = None
+ namespace = list(jnamespace) if jnamespace is not None else None
views.append(
Table(
name=jtable.name(),
@@ -694,10 +691,7 @@ class Catalog:
jtable = iter.next()
jnamespace = jtable.namespace()
- if jnamespace is not None:
- namespace = [jnamespace[i] for i in range(0, len(jnamespace))]
- else:
- namespace = None
+ namespace = list(jnamespace) if jnamespace is not None else None
tables.append(
Table(
@@ -754,10 +748,7 @@ class Catalog:
"""
jtable = self._jcatalog.getTable(tableName)
jnamespace = jtable.namespace()
- if jnamespace is not None:
- namespace = [jnamespace[i] for i in range(0, len(jnamespace))]
- else:
- namespace = None
+ namespace = list(jnamespace) if jnamespace is not None else None
return Table(
name=jtable.name(),
catalog=jtable.catalog(),
@@ -815,10 +806,7 @@ class Catalog:
while iter.hasNext():
jfunction = iter.next()
jnamespace = jfunction.namespace()
- if jnamespace is not None:
- namespace = [jnamespace[i] for i in range(0, len(jnamespace))]
- else:
- namespace = None
+ namespace = list(jnamespace) if jnamespace is not None else None
functions.append(
Function(
@@ -920,10 +908,7 @@ class Catalog:
"""
jfunction = self._jcatalog.getFunction(functionName)
jnamespace = jfunction.namespace()
- if jnamespace is not None:
- namespace = [jnamespace[i] for i in range(0, len(jnamespace))]
- else:
- namespace = None
+ namespace = list(jnamespace) if jnamespace is not None else None
return Function(
name=jfunction.name(),
catalog=jfunction.catalog(),
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
