zero323 commented on a change in pull request #34273:
URL: https://github.com/apache/spark/pull/34273#discussion_r727946428
##########
File path:
examples/src/main/python/sql/streaming/structured_network_wordcount_windowed.py
##########
@@ -87,7 +87,7 @@
windowedCounts = words.groupBy(
window(words.timestamp, windowDuration, slideDuration),
words.word
- ).count().orderBy('window')
+ ).count().orderBy('window') # type: ignore[arg-type]
Review comment:
During tests with `mypy
0.920+dev.5f7eb8161bdeefcc526705a48b03115aa1e83a14` I experienced
non-deterministic problems resulting in
```
examples/src/main/python/sql/streaming/structured_network_wordcount_windowed.py:90:
error: Argument 1 has incompatible type "str"; expected "DataFrame" [arg-type]
Found 1 error in 1 file (checked 8 source files)
```
It seems like it happens because `orderBy` is created as an alias for `sort`
https://github.com/apache/spark/blob/9290924054f8e4e24afe3c6796465c4cf60e3a2b/python/pyspark/sql/dataframe.py#L1649
and in certain conditions (?) mypy treats it as static or classmethod (could
be wrong).
I couldn't reproduce it on synthetic examples, though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
