srowen commented on a change in pull request #31254:
URL: https://github.com/apache/spark/pull/31254#discussion_r561465068
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
##########
@@ -2670,6 +2670,7 @@ class Dataset[T] private[sql](
* <li>min</li>
* <li>max</li>
* <li>arbitrary approximate percentiles specified as a percentage (e.g.
75%)</li>
+ * <li>count_distinct</li>
Review comment:
It's true, though, users looking for summary stats are probably looking
at summary(), and sometimes looking for distinct values, and this puts them
into one output with the others. I think it's slightly worth adding one more
somewhat-common summary stat.
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
##########
@@ -2670,6 +2670,7 @@ class Dataset[T] private[sql](
* <li>min</li>
* <li>max</li>
* <li>arbitrary approximate percentiles specified as a percentage (e.g.
75%)</li>
+ * <li>count_distinct</li>
Review comment:
Sure, but this is only computed if the caller requests it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
