cloud-fan commented on a change in pull request #28390:
URL: https://github.com/apache/spark/pull/28390#discussion_r417466050
##########
File path: docs/sql-migration-guide.md
##########
@@ -28,6 +28,8 @@ license: |
- In Spark 3.1, SQL UI data adopts the `formatted` mode for the query plan
explain results. To restore the behavior before Spark 3.0, you can set
`spark.sql.ui.explainMode` to `extended`.
+ - In Spark 3.1, the column metadata will always be propagated in the API
`name` and `as`. In Spark version 3.0 and earlier, the metadata of
`NamedExpression` is set as the `explicitMetadata` for the new column. To
restore the behavior before Spark 3.0, you can use the API `as(alias: String,
metadata: Metadata)` with explicit metadata.
Review comment:
> In Spark version 3.0 and earlier, the metadata of `NamedExpression` is
set as the `explicitMetadata` for the new column.
I think the old behavior is using the metadata of `NamedExpression` at the
time the API was called. The metadata won't change even if the underlying
`NamedExpression` changes metadata.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
