Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21228#discussion_r186274561
--- Diff: python/pyspark/sql/functions.py ---
@@ -152,13 +152,19 @@ def _():
_collect_list_doc = """
Aggregate function: returns a list of objects with duplicates.
+ .. note:: The function is non-deterministic because the order of
collected results depends
+ on order of rows which may be non-deterministic after a shuffle.
--- End diff --
I feel that non-deterministic here is different with other
non-deterministic like `monotonically_increasing_id` or `uuid`.
Should we just say `The order of collected results is non-deterministic and
depends on order of rows which may be non-deterministic after a shuffle`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
