This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4058a6dc7326 [SPARK-59118][SQL][FOLLOWUP] Align allowIneffectiveDoAs
version and 4.4 migration note with branch-4.x
4058a6dc7326 is described below
commit 4058a6dc7326e299372c9a00b9dba47e89b7b054
Author: Wenchen Fan <[email protected]>
AuthorDate: Wed Oct 28 03:38:25 2026 +0000
[SPARK-59118][SQL][FOLLOWUP] Align allowIneffectiveDoAs version and 4.4
migration note with branch-4.x
### What changes were proposed in this pull request?
A follow-up to #58414, now that its `branch-4.x` backport #58438 has
shipped `spark.sql.hive.thriftServer.allowIneffectiveDoAs` in 4.4.0 (as a
warn-only gate):
- Change the conf's `.version("5.0.0")` to `.version("4.4.0")`, since 4.4.0
is the first release with this key.
- Add the backport's 4.4 migration note to the "Upgrading from Spark SQL
4.3 to 4.4" section, at the same position as on `branch-4.x`. That section is
otherwise identical between `master` and `branch-4.x`, and the 5.0 note added
by #58414 already says "Spark 4.4 logs a warning for the same configuration."
### Why are the changes needed?
A conf's version should record the first release that has it, and master's
4.3 to 4.4 migration section should match `branch-4.x`'s.
### Does this PR introduce _any_ user-facing change?
No. Behavior is unchanged; the generated conf docs list the conf as
available since 4.4.0 instead of 5.0.0.
### How was this patch tested?
No new tests: the change is a version string and one documentation line.
Checked that master's "Upgrading from Spark SQL 4.3 to 4.4" section is now
identical to `branch-4.x`'s.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Isaac (Claude Opus 5.5)
This pull request and its description were written by Isaac.
Closes #59831 from cloud-fan/wenchen/spark-59118-doas-version-4.4.
Authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
docs/sql-migration-guide.md | 1 +
.../src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 8d62c4a2dfac..8585804bba4f 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -61,6 +61,7 @@ license: |
- Since Spark 4.4, with whole-stage codegen enabled, a filter condition,
including the `FILTER` clause of an aggregate function, evaluates the remaining
top-level conjuncts of the condition on a row after one of them evaluates to
`NULL`, as the SQL standard requires, and does so in the order they appear in
the condition, as Spark already did with whole-stage codegen disabled, which
also evaluates a subexpression shared by several conjuncts before the
condition. Only a `FALSE` conjunct st [...]
- Since Spark 4.4, when a format is specified, `to_timestamp`,
`try_to_timestamp`, and `to_timestamp_ltz` cast a numeric first argument to a
string and parse it with the format, for example `to_timestamp(20201231,
'yyyyMMdd')` returns `2020-12-31 00:00:00`. This matches the existing behavior
when `spark.sql.timestampType` is `TIMESTAMP_NTZ`. Previously, with the default
`TIMESTAMP_LTZ` timestamp type, such calls failed with `INTERNAL_ERROR`. Note
that the string form of the number is par [...]
+- Since Spark 4.4, the Spark Thrift Server logs a warning at startup when
`hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies
user identities (any recognized type other than `NONE`/`NOSASL`), because
executor-side data access still runs as the server's own service identity
rather than the connecting user (SPARK-5159). Set
`spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true` to acknowledge the
limitation and silence the warning. Spark 5.0 is expected to [...]
## Upgrading from Spark SQL 4.2 to 4.3
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
index 74ebf46b0861..ac5e7449b37e 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
@@ -158,7 +158,7 @@ object StaticSQLConf {
"server refuses to start when hive.server2.enable.doAs=true and " +
"hive.server2.authentication verifies user identities (any recognized
type other " +
"than NONE/NOSASL), unless this option is set to true to acknowledge
the limitation.")
- .version("5.0.0")
+ .version("4.4.0")
.withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
.booleanConf
.createWithDefault(false)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
