Yikun commented on code in PR #37521:
URL: https://github.com/apache/spark/pull/37521#discussion_r952062535
##########
python/pyspark/sql/column.py:
##########
@@ -495,6 +544,17 @@ def dropFields(self, *fieldNames: str) -> "Column":
.. versionadded:: 3.1.0
+ Parameters
+ ----------
+ fieldNames : str or list
Review Comment:
```suggestion
fieldNames : str
```
`list` is not accurate, it's just a positional arguments.
```
>>> df.withColumn('a', df['a'].dropFields('b', 'c')).show()
+--------------+
| a|
+--------------+
|{3, {4, 5, 6}}|
+--------------+
>>> df.withColumn('a', df['a'].dropFields(['b', 'c'])).show()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/yikun/spark/python/pyspark/sql/column.py", line 543, in
dropFields
jc = self._jc.dropFields(_to_seq(sc, fieldNames))
File
"/Users/yikun/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py",
line 1322, in __call__
File "/Users/yikun/spark/python/pyspark/sql/utils.py", line 190, in deco
return f(*a, **kw)
File
"/Users/yikun/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/protocol.py", line
326, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o88.dropFields.
: java.lang.ClassCastException: java.util.ArrayList cannot be cast to
java.lang.String
```
I think below is a good example to describe positional arguments:
https://github.com/apache/spark/blob/af2d2ee11530a98e748518a81925f3f0a2392209/python/pyspark/sql/column.py#L861
##########
python/pyspark/sql/column.py:
##########
@@ -462,6 +499,18 @@ def withField(self, fieldName: str, col: "Column") ->
"Column":
.. versionadded:: 3.1.0
+ Parameters
+ ----------
+ fieldName : str
+ a literal value.
+ The result will only be true at a location if any field matches in
the Column.
Review Comment:
`col` parameter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
