Spenserrrr commented on code in PR #57702:
URL: https://github.com/apache/spark/pull/57702#discussion_r3706802170
##########
python/pyspark/sql/tests/connect/test_connect_basic.py:
##########
@@ -842,6 +842,15 @@ def test_select_expr(self):
.toPandas(),
)
+ self.assert_eq(
+ self.connect.read.table(self.tbl_name)
+ .selectExpr(("id * 2", "cast(name as long) as name"))
Review Comment:
Thank you so much for taking a look, @HyukjinKwon!
This is one of the three related PRs on cleaning up the pattern that varargs
also accepts a single sequence, so I think this question applies to all three
as well. The other two PR: #57693, #57726.
Also, these PRs are suggested by Tian's comment on #57611, and he suggested
handling this pattern at the repo level, and he pointed me to his #54764 as a
reference. He widened `List` to `Sequence` and added the two overloads. So
that's the pattern I followed here to make these consistent with what is
already in `dataframe.py`.
For how widespread this is, on the current master, the single-collection
form is already accepted at around a dozen places. For example,
partitionBy/clusterBy on the writers and stream writers (classic and connect),
and struct/create_map/array/map_concat, which takes a list or a set (and in
connect a tuple too, so a tuple works in connect but not classic for the same
public function today). And in dataframe.py,
select/groupBy/rollup/cube/sort/sortWithinPartitions already carry the
Sequence[...] annotation and the two overloads from Tian's edit.
Also, a quick summary of my PRs: #57693 only corrects annotations for the
writer methods, which already accept list and tuple; this PR and #57726 widen
list-only sites, which do add tuple support; and I plan to create another PR
for struct/create_map/array/map_concat because a set isn't a Sequence and
classic/connect disagree there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
