This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 68d6ad186c43 [SPARK-57775][SQL][FOLLOWUP] Document the grand-total
behavior change in the migration guide
68d6ad186c43 is described below
commit 68d6ad186c436296c4e6b0851851c696130b2774
Author: Uros Bojanic <[email protected]>
AuthorDate: Wed Jul 8 09:14:12 2026 -0700
[SPARK-57775][SQL][FOLLOWUP] Document the grand-total behavior change in
the migration guide
### What changes were proposed in this pull request?
Follow-up to #56891: add a SQL migration guide entry for the behavior
change: `GROUP BY GROUPING SETS (())` and empty `CUBE()` / `ROLLUP()` now
return one row over empty input.
### Why are the changes needed?
The behavior change was requested to be documented in the migration guide
as part of
https://github.com/apache/spark/pull/56891#pullrequestreview-4649357419.
### Does this PR introduce _any_ user-facing change?
No, docs only change.
### How was this patch tested?
N/A, docs only change.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57121 from uros-b/SPARK-57775-migration-guide.
Lead-authored-by: Uros Bojanic <[email protected]>
Co-authored-by: Uros Bojanic <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/sql-migration-guide.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 460a4d4219bf..a0a7057e51f7 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -27,6 +27,7 @@ license: |
- Since Spark 4.3, zero-length files are skipped during Parquet schema
inference instead of failing with a `FAILED_READ_FILE.CANNOT_READ_FILE_FOOTER`
error.
- Since Spark 4.3, the configuration key
`spark.sql.sources.v2.bucketing.allowJoinKeysSubsetOfPartitionKeys.enabled` has
been renamed to
`spark.sql.sources.v2.bucketing.allowKeysSubsetOfPartitionKeys.enabled` to
reflect that it now applies to storage-partitioned joins, aggregates, and
windows. The old key continues to work as an alias.
- Since Spark 4.3, the Spark Thrift Server rejects setting JVM system
properties through the `set:system:` session configuration overlay (for
example, in a JDBC connection string). To restore the previous behavior, set
`spark.sql.legacy.hive.thriftServer.allowSettingSystemProperties` to `true`.
+- Since Spark 4.3, `GROUP BY GROUPING SETS (())` (and the equivalent empty
`GROUP BY CUBE()` / `GROUP BY ROLLUP()`) is treated as a grand total and
returns one row over empty input, matching an aggregation with no `GROUP BY`
clause; previously it returned no rows. To restore the previous behavior, set
`spark.sql.analyzer.lowerEmptyGroupingSetToGlobalAggregate.enabled` to `false`.
## Upgrading from Spark SQL 4.1 to 4.2
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
