marmbrus commented on a change in pull request #28996:
URL: https://github.com/apache/spark/pull/28996#discussion_r453046549
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
##########
@@ -2030,7 +2030,25 @@ class Dataset[T] private[sql](
* @group typedrel
* @since 2.3.0
*/
- def unionByName(other: Dataset[T]): Dataset[T] = withSetOperator {
+ def unionByName(other: Dataset[T]): Dataset[T] = unionByName(other, false)
+
+ /**
+ * Returns a new Dataset containing union of rows in this Dataset and
another Dataset.
+ *
+ * This is different from both `UNION ALL` and `UNION DISTINCT` in SQL. To
do a SQL-style set
+ * union (that does deduplication of elements), use this function followed
by a [[distinct]].
Review comment:
Wait really? When did we change the semantics? What was confusing about
that documentation? (it was added because users were confused by the
behavior...)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
