dongjoon-hyun commented on code in PR #39470:
URL: https://github.com/apache/spark/pull/39470#discussion_r1065146578
##########
docs/sql-migration-guide.md:
##########
@@ -35,6 +35,7 @@ license: |
- Valid values for `fmt` are case-insensitive `hex`, `base64`, `utf-8`,
`utf8`.
- Since Spark 3.4, Spark throws only `PartitionsAlreadyExistException` when
it creates partitions but some of them exist already. In Spark 3.3 or earlier,
Spark can throw either `PartitionsAlreadyExistException` or
`PartitionAlreadyExistsException`.
- Since Spark 3.4, Spark will do validation for partition spec in ALTER
PARTITION to follow the behavior of `spark.sql.storeAssignmentPolicy` which may
cause an exception if type conversion fails, e.g. `ALTER TABLE .. ADD
PARTITION(p='a')` if column `p` is int type. To restore the legacy behavior,
set `spark.sql.legacy.skipTypeValidationOnAlterPartition` to `true`.
+ - Since Spark 3.4, vectorized readers are enabled by default for the nested
data types (array, map and struct). To restore the legacy behavior, set
`spark.sql.orc.enableNestedColumnVectorizedReader` and
`spark.sql.parquet.enableNestedColumnVectorizedReader` to `false`.
Review Comment:
This is #36124 and #39434 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
