zero323 commented on a change in pull request #34354:
URL: https://github.com/apache/spark/pull/34354#discussion_r779093510
##########
File path: python/pyspark/sql/functions.py
##########
@@ -1670,7 +1671,19 @@ def expr(str: str) -> Column:
return Column(sc._jvm.functions.expr(str))
+@overload
def struct(*cols: "ColumnOrName") -> Column:
+ ...
+
+
+@overload
+def struct(__cols: Union[List["ColumnOrName_"], Tuple["ColumnOrName_", ...]])
-> Column:
+ ...
Review comment:
We still want to support calls like
```python
struct(("foo", "bar"))
```
which shouldn't be accepted without `Tuple` (or some supertype).
If you try
```python
diff --git a/python/pyspark/sql/functions.py
b/python/pyspark/sql/functions.py
index 006d10c9fc..caf17a84b3 100644
--- a/python/pyspark/sql/functions.py
+++ b/python/pyspark/sql/functions.py
@@ -1677,13 +1677,11 @@ def struct(*cols: "ColumnOrName") -> Column:
@overload
-def struct(__cols: Union[List["ColumnOrName_"], Tuple["ColumnOrName_",
...]]) -> Column:
+def struct(__cols: Union[List["ColumnOrName_"]]) -> Column:
...
-def struct(
- *cols: Union["ColumnOrName", Union[List["ColumnOrName_"],
Tuple["ColumnOrName_", ...]]]
-) -> Column:
+def struct(*cols: Union["ColumnOrName", Union[List["ColumnOrName_"]]]) ->
Column:
"""Creates a new struct column.
.. versionadded:: 1.4.0
diff --git a/python/pyspark/sql/tests/typing/test_functions.yml
b/python/pyspark/sql/tests/typing/test_functions.yml
index efb3293472..fdf2303890 100644
--- a/python/pyspark/sql/tests/typing/test_functions.yml
+++ b/python/pyspark/sql/tests/typing/test_functions.yml
@@ -44,6 +44,7 @@
struct([col("foo"), col("bar")])
struct("foo", "bar")
struct(["foo", "bar"])
+ struct(("foo", "bar"))
array([col("foo")], [col("bar")])
create_map([col("foo")], [col("bar")])
```
you should see error in data tests.
> And could I ask if what is the `...` mean in `["ColumnOrName_", ...]` ??
`Tuples` are typed like product types, so `Tuple[ColumnOrName_]` matches
tuple with exactly one column or str element. In contrast
`Tuple["ColumnOrName_", ...]` matches tuples of arbitrary size, as long as all
elements are columns or strings (there is [mypy doc
section](https://mypy.readthedocs.io/en/stable/kinds_of_types.html?highlight=namedtuple#tuple-types)
that discusses this further).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
