Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21228#discussion_r186281087
--- Diff: python/pyspark/sql/functions.py ---
@@ -152,13 +152,19 @@ def _():
_collect_list_doc = """
Aggregate function: returns a list of objects with duplicates.
+ .. note:: The function is non-deterministic because the order of
collected results depends
+ on order of rows which may be non-deterministic after a shuffle.
--- End diff --
Nature of non-determinism can be different but I believe it is important to
mention explicitly in the note that the function is `non-deterministic`. It
would be common style of notes for all non-deterministic functions. Maybe it is
not good example but event for `grep`ing or searching in docs it is pretty
convenient.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
