szehon-ho commented on code in PR #46673:
URL: https://github.com/apache/spark/pull/46673#discussion_r1607370785
##########
docs/sql-migration-guide.md:
##########
@@ -55,6 +55,7 @@ license: |
- Since Spark 4.0, The default value for `spark.sql.legacy.timeParserPolicy`
has been changed from `EXCEPTION` to `CORRECTED`. Instead of raising an
`INCONSISTENT_BEHAVIOR_CROSS_VERSION` error, `CANNOT_PARSE_TIMESTAMP` will be
raised if ANSI mode is enable. `NULL` will be returned if ANSI mode is
disabled. See [Datetime Patterns for Formatting and
Parsing](sql-ref-datetime-pattern.html).
- Since Spark 4.0, A bug falsely allowing `!` instead of `NOT` when `!` is not
a prefix operator has been fixed. Clauses such as `expr ! IN (...)`, `expr !
BETWEEN ...`, or `col ! NULL` now raise syntax errors. To restore the previous
behavior, set `spark.sql.legacy.bangEqualsNot` to `true`.
- Since Spark 4.0, Views allow control over how they react to underlying query
changes. By default views tolerate column type changes in the query and
compensate with casts. To restore the previous behavior, allowing up-casts
only, set `spark.sql.viewSchemaBindingMode` to `DISABLED`. This disables the
feature and also disallows the `WITH SCHEMA` clause.
+- Since Spark 4.0, The Storage-Partitioned Join feature flag
`spark.sql.sources.v2.bucketing.enabled` and
`spark.sql.sources.v2.bucketing.pushPartValues.enabled` is set to `true`. To
restore the previous behavior, set `spark.sql.sources.v2.bucketing.enabled` and
`spark.sql.sources.v2.bucketing.pushPartValues.enabled` to `false`.
Review Comment:
Ok I took this from @superdiaodiao , who also worked on this pr, will revert.
##########
sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala:
##########
@@ -1169,7 +1169,6 @@ class KeyGroupedPartitioningSuite extends
DistributionAndOrderingSuiteBase {
Seq(true, false).foreach { shuffle =>
withSQLConf(
SQLConf.V2_BUCKETING_SHUFFLE_ENABLED.key -> shuffle.toString,
- SQLConf.V2_BUCKETING_PUSH_PART_VALUES_ENABLED.key -> "true",
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
