cloud-fan commented on a change in pull request #33040:
URL: https://github.com/apache/spark/pull/33040#discussion_r657226872
##########
File path: docs/sql-migration-guide.md
##########
@@ -96,6 +96,8 @@ license: |
- In Spark 3.2, `FloatType` is mapped to `FLOAT` in MySQL. Prior to this, it
used to be mapped to `REAL`, which is by default a synonym to `DOUBLE
PRECISION` in MySQL.
- In Spark 3.2, the query executions triggered by `DataFrameWriter` are
always named `command` when being sent to `QueryExecutionListener`. In Spark
3.1 and earlier, the name is one of `save`, `insertInto`, `saveAsTable`.
+
+ - In Spark 3.2, `Dataset.unionByName` with `allowMissingColumns` set to true
will add missing nested fields to the end of structs. In Spark 3.1, nested
struct fields are sorted alphabetically.
Review comment:
union of top-level columns is also "left dominant", this makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
