This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new fb51de5959f5 [SPARK-60757][SQL] Do not apply a writer's session 
settings to the generation expression of a generated column
fb51de5959f5 is described below

commit fb51de5959f5abbf40874bd6a2bd502a5676b133
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Nov 3 21:13:20 2026 -0800

    [SPARK-60757][SQL] Do not apply a writer's session settings to the 
generation expression of a generated column
    
    ### What changes were proposed in this pull request?
    
    This PR makes a write read the stored text of a generation expression under 
fixed settings, instead of the settings of the session that writes.
    
    - `StoredExpressionText` is a new helper that defines those settings: the 
default `SQLConf`, the configs that 
`Analyzer.retainResolutionConfigsForAnalysis` keeps, and ANSI enabled. The 
session time zone, case sensitivity, `spark.sql.timestampType` and 
`spark.sql.decimalOperations.allowPrecisionLoss` stay with the session, because 
the settings of the DDL are not stored and pinning them would silently change 
existing tables. Any other config that the parsing or the analysis reads takes  
[...]
    - `TableOutputResolver`, which auto-fills a generated column, and 
`ResolveTableConstraints.buildGenerationCatalystExpression`, which validates a 
provided value, parse the text under those settings with `CatalystSqlParser`. 
The validation used to convert the V2 expression back first, which carries the 
implicit casts of the session of the DDL. It now uses the V2 expression only 
when there is no text, and the `GenerationExpression` and `Column` Javadocs say 
so.
    - `ResolveOutputRelation` analyzes the auto-filled columns under those 
settings, and `ResolveTableConstraints` analyzes the check of a provided value 
under them. Implicit casts and the ANSI behavior of functions are decided 
during analysis, so parsing alone would leave them to the writer. Using the 
same settings on both sides keeps a row that Spark auto-filled valid when a 
later write carries it. The check also casts the expression to the column type 
with an ANSI cast, as store assign [...]
    - `ResolveTableConstraints` skips the check of generated columns for a 
row-level `DELETE`, which writes no new generated value.
    - `GeneratedColumnExpression.validate` rejects a generation expression that 
reads differently under those settings than in the session that creates the 
table, with a new reason of `UNSUPPORTED_EXPRESSION_GENERATED_COLUMN` that 
names the parser configs. The comparison uses the ANSI value of the session, so 
it is only about how the text reads.
    - `spark.sql.legacy.useCurrentConfigsForGeneratedColumn` (internal, default 
`false`, `SESSION` binding policy) reads the text under the session of the 
write, as Apache Spark 4.3.0 does, and turns off the rejection above. The other 
changes of this PR stay, such as reading the text before the V2 expression, the 
cast in the check, and skipping a `DELETE`.
    - `docs/sql-migration-guide.md` describes the behavior changes.
    
    ### Why are the changes needed?
    
    Generated column values on V2 table writes were added by SPARK-57644 and 
first released in Apache Spark 
[v4.3.0](https://github.com/apache/spark/releases/tag/v4.3.0) (2026-10-27). The 
value of a generated column depends on the session that writes it, and a wrong 
value is stored silently.
    
    ```sql
    CREATE TABLE t (a STRING, b STRING GENERATED ALWAYS AS (concat(a, 'x\\y')));
    INSERT INTO t (a) VALUES ('default');  -- b = defaultx\y
    SET spark.sql.parser.escapedStringLiterals=true;
    INSERT INTO t (a) VALUES ('escaped');  -- b = escapedx\\y
    INSERT INTO t (a, b) SELECT 'p', concat('p', 'x', chr(92), 'y');  -- 
CHECK_CONSTRAINT_VIOLATION
    INSERT INTO t (a, b) SELECT 'q', concat('q', 'x', chr(92), chr(92), 'y');  
-- succeeds
    ```
    
    ```sql
    CREATE TABLE t (a STRING, b BIGINT GENERATED ALWAYS AS (a + 1));
    SET spark.sql.ansi.enabled=false;
    INSERT INTO t (a) VALUES ('abc');  -- b = NULL
    INSERT INTO t (a) VALUES ('9007199254740993');  -- b = 9007199254740992
    ```
    
    After this PR, the first example stores `escapedx\y`, accepts `p`, and 
rejects `q`. The second example fails with `CAST_INVALID_INPUT` for `abc` and 
stores `9007199254740994`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, compared to Apache Spark 4.3.0.
    
    - The settings of the writer no longer change an auto-filled value or the 
validation of a provided value through the parser settings, the ANSI mode, or a 
temporary function that replaces a built-in function.
    - A write with `spark.sql.ansi.enabled=false` now fails with the ANSI error 
of a generation expression instead of storing `NULL`, also for a table created 
with that setting. For example, `b DOUBLE GENERATED ALWAYS AS (a + 1)` with `a 
= '1.5'` used to store `2.5` when both the DDL and the write ran with that 
setting, and now fails with `CAST_INVALID_INPUT`. An implicit cast can also 
change a stored value. For example, `b STRING GENERATED ALWAYS AS (coalesce(s, 
1))` with `s = '007'` use [...]
    - A table created with a non-default value of a config such as 
`spark.sql.parser.escapedStringLiterals` gets the default reading of its text 
on every write, without an error. So does a config that a function reads while 
it is built. For example, with 
`spark.sql.legacy.negativeIndexInArrayInsert=true`, `array_insert(a, -1, 'x')` 
used to store `[a, x, b]` for `a = [a, b]`, and now stores `[a, b, x]`.
    - A row that was stored that way can fail when a later write provides it 
again, such as `INSERT INTO t SELECT * FROM t`. For example, with the `DOUBLE` 
column above, a stored `b = 9.007199254740992E15` for `a = '9007199254740993'` 
fails with `CHECK_CONSTRAINT_VIOLATION`, and a stored `b = NULL` for `a = 
'abc'` fails with `CAST_INVALID_INPUT`. Such a write already fails the same way 
in Apache Spark 4.3.0 when it runs with ANSI enabled.
    - A `DELETE` no longer checks the rows that it carries, so a row that was 
stored that way does not fail it. In Apache Spark 4.3.0, a `DELETE` that 
rewrites rows fails on such a row when it runs with ANSI enabled.
    - The validation of a provided value compares it with the value that the 
write would store, so a value that only casts to it now fails with 
`CHECK_CONSTRAINT_VIOLATION`, also with ANSI enabled. For example, for `b 
STRING GENERATED ALWAYS AS (coalesce(s, 1))` with `s = '007'`, which stores 
`7`, a provided `07` used to pass with ANSI enabled and now fails.
    - A table CHECK constraint is still read under the session that writes. So 
a constraint that repeats the text of a generation expression can fail a write 
with a non-default parser setting, such as 
`spark.sql.parser.escapedStringLiterals=true`, where Apache Spark 4.3.0 stored 
the value that the setting gives.
    - The V2 expression that `CREATE TABLE` gives a catalog still comes from 
the analysis in the session of the DDL. With `spark.sql.ansi.enabled=false`, it 
can differ from a value that a write stores. For example, it is `COALESCE(s, 
'1')` for the `coalesce(s, 1)` column above. Spark itself uses it only when 
there is no text.
    - `CREATE TABLE` and `REPLACE TABLE` reject a generation expression that 
the session reads differently. For example, with 
`spark.sql.parser.escapedStringLiterals=true`, `concat(a, 'x\\y')` is rejected, 
while `concat(a, 'xy')` is accepted. With 
`spark.sql.ansi.enforceReservedKeywords=true`, a bare `current_date` is 
rejected, and `curdate()` is accepted. This also applies to a table without the 
`GENERATE_COLUMN_VALUES_ON_WRITE` capability.
    - `spark.sql.legacy.useCurrentConfigsForGeneratedColumn=true` reads the 
text under the session of the write, as Apache Spark 4.3.0 does, and restores 
the previous DDL behavior.
    
    The following still depend on the session that writes:
    
    - A generation expression that the fixed settings cannot resolve or 
analyze. It keeps their parser settings, and its ANSI behavior and analysis 
follow the session as in Apache Spark 4.3.0. A text that they cannot parse is 
parsed under the session. For example, `flag = 1` on a `BOOLEAN` column works 
with `spark.sql.ansi.enabled=false` and fails with 
`DATATYPE_MISMATCH.BINARY_OP_DIFF_TYPES` with ANSI enabled. `CAST(d AS 
BOOLEAN)` on a `DATE` column stores `NULL` with ANSI disabled and f [...]
    - The functions found through `SET PATH`.
    - The session time zone, for example for `hour(ts)`.
    - `spark.sql.timestampType`, for example for `to_timestamp(a)` and `CAST(a 
AS TIMESTAMP)`.
    - `spark.sql.decimalOperations.allowPrecisionLoss`, for example for `a * a` 
on `DECIMAL` columns.
    - An ANSI cast that `ConstantFolding` removes under a session with ANSI 
disabled, such as in `size(array(CAST(a AS INT)))`, as in Apache Spark 4.3.0.
    - The settings that an expression reads during evaluation, such as 
`spark.sql.legacy.timeParserPolicy` and 
`spark.sql.legacy.castComplexTypesToString.enabled`.
    
    ### How was this patch tested?
    
    Added test cases to `GeneratedColumnWriteSuite`, updated one test case in 
`DataSourceV2DataFrameSuite`, and changed the placeholder text in 
`GeneratedColumnExpressionSuite`. Eighteen of the new test cases fail without 
this PR.
    
    Ran locally:
    
    ```bash
    build/sbt 'catalyst/testOnly *GeneratedColumnExpressionSuite'
    build/sbt 'sql/testOnly *GeneratedColumnWriteSuite 
*DataSourceV2DataFrameSuite *CheckConstraintSuite *DeleteFromTable*'
    SPARK_ANSI_SQL_MODE=false build/sbt 'sql/testOnly 
*GeneratedColumnWriteSuite *DataSourceV2DataFrameSuite'
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    This patch had conflicts when merged, resolved by
    Committer: Dongjoon Hyun <[email protected]>
    
    Closes #59965 from dongjoon-hyun/SPARK-60757.
    
    Authored-by: Dongjoon Hyun <[email protected]>
    Signed-off-by: Dongjoon Hyun <[email protected]>
    (cherry picked from commit 6e287bf9bf3bb6c6e464c53350db507b210dfd1e)
    Signed-off-by: Dongjoon Hyun <[email protected]>
---
 docs/sql-migration-guide.md                        |   2 +
 .../apache/spark/sql/connector/catalog/Column.java |   8 +-
 .../connector/catalog/GenerationExpression.java    |   7 +-
 .../spark/sql/catalyst/analysis/Analyzer.scala     |  42 +-
 .../analysis/ResolveTableConstraints.scala         |  67 ++-
 .../catalyst/analysis/TableOutputResolver.scala    |  27 +-
 .../plans/logical/GeneratedColumnExpression.scala  |  13 +-
 .../sql/catalyst/util/StoredExpressionText.scala   | 142 +++++
 .../spark/sql/errors/QueryCompilationErrors.scala  |   4 +
 .../org/apache/spark/sql/internal/SQLConf.scala    |  14 +
 .../logical/GeneratedColumnExpressionSuite.scala   |   2 +-
 .../sql/connector/DataSourceV2DataFrameSuite.scala |  37 +-
 .../sql/connector/GeneratedColumnWriteSuite.scala  | 572 ++++++++++++++++++++-
 13 files changed, 882 insertions(+), 55 deletions(-)

diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index edb882b73634..2440185e1653 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -69,6 +69,8 @@ license: |
 - Since Spark 4.4, the Spark Thrift Server logs a warning at startup when 
`hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies 
user identities (any recognized type other than `NONE`/`NOSASL`), because 
executor-side data access still runs as the server's own service identity 
rather than the connecting user (SPARK-5159). Set 
`spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true` to acknowledge the 
limitation and silence the warning. Spark 5.0 is expected to [...]
 - Since Spark 4.4, `UPDATE` and `MERGE` on a Data Source V2 table fail with 
`DATATYPE_MISMATCH.INVALID_ROW_LEVEL_OPERATION_ASSIGNMENTS` when they assign to 
a metadata column of the table. This includes `UPDATE SET *` and `INSERT *` in 
`MERGE WITH SCHEMA EVOLUTION` when the source has a column with the name of a 
metadata column of the target table. Previously, these assignments passed 
analysis and were usually ignored. To restore the previous behavior, set 
`spark.sql.legacy.ignoreMetadata [...]
 - Since Spark 4.4, `CREATE TABLE`, `REPLACE TABLE`, and `ALTER TABLE ... ADD 
CONSTRAINT` on a Data Source V2 table fail with 
`USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT` when the condition of a CHECK 
constraint calls a user-defined scalar function, including a Data Source V2 
function. `ALTER TABLE ... ADD CONSTRAINT` keeps failing with 
`UNSUPPORTED_SQL_UDF_USAGE` on a SQL function. Previously, a write could 
resolve the function name in its own session, so that it enforced another rule 
or  [...]
+- Since Spark 4.4, a write to a Data Source V2 table parses and analyzes the 
stored text of a generation expression with ANSI mode on and the default values 
of the SQL configs that change how a text reads, such as 
`spark.sql.parser.escapedStringLiterals`, regardless of the session of the 
write. Only the following still follow that session: the session time zone, 
case sensitivity, `spark.sql.timestampType`, 
`spark.sql.decimalOperations.allowPrecisionLoss`, the configs that a view also 
tak [...]
+- Since Spark 4.4, `CREATE TABLE` and `REPLACE TABLE` fail with 
`UNSUPPORTED_EXPRESSION_GENERATED_COLUMN` on a generation expression that the 
session reads differently than a write does, for example with 
`spark.sql.parser.escapedStringLiterals=true` or 
`spark.sql.ansi.enforceReservedKeywords=true`. This also applies to a table 
without the `GENERATE_COLUMN_VALUES_ON_WRITE` capability, which computes the 
values itself (SPARK-60757). To restore the previous behavior, set 
`spark.sql.legacy.u [...]
 - Since Spark 4.4, `CREATE TABLE` and `REPLACE TABLE` on a Data Source V2 
table fail with `AGGREGATE_OR_WINDOW_FUNCTION_IN_CHECK_CONSTRAINT` when the 
condition of a CHECK constraint calls an aggregate or window function, 
including a user-defined aggregate function, `grouping`, and `grouping_id`. 
`ALTER TABLE ... ADD CONSTRAINT` keeps failing with its existing errors, such 
as `INVALID_WHERE_CONDITION` and `WINDOW_FUNCTION_NOT_ALLOWED_IN_CLAUSE`. 
Previously, these statements succeeded and  [...]
 - Since Spark 4.4, `CREATE TABLE` and `REPLACE TABLE` on a Data Source V2 
table fail with `GENERATOR_IN_CHECK_CONSTRAINT` when the condition of a CHECK 
constraint calls a generator function, such as `explode`, `posexplode`, and 
`inline`. `ALTER TABLE ... ADD CONSTRAINT` keeps failing with 
`UNSUPPORTED_GENERATOR.OUTSIDE_SELECT`. Previously, these statements succeeded 
and every later write to the table failed (SPARK-60821). To run these 
statements, remove the function from the condition. E [...]
 - Since Spark 4.4, `ALTER TABLE ... ADD CONSTRAINT` with a CHECK constraint 
fails with `EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE` when the target is a view, as 
it does with the other constraint types. Previously, when the view only 
projected or filtered a Data Source V2 table, the statement added the 
constraint to that table after validating only the rows of the view 
(SPARK-60822). To add the constraint, run the statement on the table.
diff --git 
a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/Column.java 
b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/Column.java
index 438b64dbaf21..bf37787b5082 100644
--- 
a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/Column.java
+++ 
b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/Column.java
@@ -171,10 +171,10 @@ public interface Column {
    * Returns the generation expression of this table column as a SQL string. 
Null means no
    * generation expression.
    * <p>
-   * This returns only the SQL string form. Prefer {@link 
#columnGenerationExpression()}, which can
-   * also carry a connector {@link 
org.apache.spark.sql.connector.expressions.Expression} and
-   * captures the semantics unambiguously. It is up to the data source to 
verify expression
-   * compatibility and reject writes as necessary.
+   * This returns only the SQL string form. {@link 
#columnGenerationExpression()} can also carry a
+   * connector {@link org.apache.spark.sql.connector.expressions.Expression}. 
See
+   * {@link GenerationExpression} for how Spark reads each form. It is up to 
the data source to
+   * verify expression compatibility and reject writes as necessary.
    */
   @Nullable
   default String generationExpression() {
diff --git 
a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/GenerationExpression.java
 
b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/GenerationExpression.java
index 8e64eba9c516..0395620f53d6 100644
--- 
a/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/GenerationExpression.java
+++ 
b/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/GenerationExpression.java
@@ -26,9 +26,10 @@ import org.apache.spark.sql.connector.expressions.Expression;
  * Connectors can define generated columns using either a SQL string (Spark 
SQL dialect) or a more
  * portable connector {@link Expression expression}, whose semantics follow 
the ANSI SQL standard.
  * <p>
- * If both the SQL string and the expression are provided, Spark first 
attempts to convert the
- * given expression to its internal representation. If the expression cannot 
be converted, and a
- * SQL string is provided, Spark will fall back to parsing the SQL string.
+ * Spark computes a generated value on a write only from the SQL string, so a 
column without one is
+ * not auto-filled. To check a provided value, Spark parses the SQL string if 
one is provided, also
+ * when an expression is provided. Otherwise, Spark converts the given 
expression to its internal
+ * representation.
  *
  * @since 4.3.0
  */
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index ef6d53cc3a9c..4ffa9c1d8117 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -48,7 +48,7 @@ import org.apache.spark.sql.catalyst.trees.AlwaysProcess
 import org.apache.spark.sql.catalyst.trees.CurrentOrigin.withOrigin
 import org.apache.spark.sql.catalyst.trees.TreePattern._
 import org.apache.spark.sql.catalyst.types.DataTypeUtils
-import org.apache.spark.sql.catalyst.util.{toPrettySQL, 
trimTempResolvedColumn, CharVarcharUtils, GeneratedColumn}
+import org.apache.spark.sql.catalyst.util.{toPrettySQL, 
trimTempResolvedColumn, CharVarcharUtils, GeneratedColumn, StoredExpressionText}
 import org.apache.spark.sql.catalyst.util.ResolveDefaultColumns._
 // `View` is aliased to `V2View` to avoid clashing with the logical-plan 
`View` imported via
 // `org.apache.spark.sql.catalyst.plans.logical._`.
@@ -715,7 +715,7 @@ class Analyzer(
       ResolveWindowFrame ::
       ResolveNaturalAndUsingJoin ::
       ResolveOutputRelation ::
-      new ResolveTableConstraints(catalogManager) ::
+      new ResolveTableConstraints(catalogManager, plan => execute(plan)) ::
       new ResolveSetVariable(catalogManager) ::
       new ResolveFetchCursor(catalogManager) ::
       new ResolveCursors() ::
@@ -4123,10 +4123,12 @@ class Analyzer(
             table.schemaAlignmentConfig()
           case _ => SchemaAlignmentConfig.DEFAULT
         }
-        val (projection, autoFilledGenCols) =
+        def resolveOutput(readGenerationExpr: String => Expression) =
           TableOutputResolver.resolveOutputColumnsWithGeneratedInfo(
             v2Write.table.name, expected, v2Write.query, v2Write.isByName, 
conf,
-            defaultValueFillMode, 
schemaAlignmentConfig.ansiStoreAssignmentCastCheck())
+            defaultValueFillMode, 
schemaAlignmentConfig.ansiStoreAssignmentCastCheck(),
+            readGenerationExpr)
+        val (projection, autoFilledGenCols) = 
resolveOutput(StoredExpressionText.parse)
         if (projection != v2Write.query) {
           val cleanedTable = v2Write.table match {
             case r: DataSourceV2Relation =>
@@ -4137,11 +4139,41 @@ class Analyzer(
               cleanedRelation
             case other => other
           }
-          v2Write.withNewQuery(projection).withNewTable(cleanedTable)
+          v2Write
+            .withNewQuery(analyzeAutoFilledGeneratedColumns(
+              projection,
+              autoFilledGenCols,
+              resolveOutput(StoredExpressionText.parseWithSessionAnsi)._1))
+            .withNewTable(cleanedTable)
         } else {
           v2Write
         }
     }
+
+    /**
+     * Analyzes the auto-filled generated columns of a write under the 
settings their stored text
+     * is read under, so that the implicit casts and the functions of a 
generation expression do
+     * not follow the session that writes.
+     *
+     * A column these settings cannot resolve comes from `readInSession`, 
where its text is read
+     * with the ANSI value of the session, and the analysis of the write 
resolves it under the
+     * session. So does a text a catalog stores that analyzes to another plan.
+     */
+    private def analyzeAutoFilledGeneratedColumns(
+        projection: LogicalPlan,
+        autoFilled: Set[String],
+        readInSession: => LogicalPlan): LogicalPlan = projection match {
+      case p @ Project(projectList, child) if autoFilled.nonEmpty =>
+        lazy val sessionCols = readInSession.expressions.collect {
+          case c: NamedExpression => c.name -> c
+        }.toMap
+        p.copy(projectList = projectList.map {
+          case c if autoFilled.contains(c.name) =>
+            StoredExpressionText.analyze(c, sessionCols(c.name), child.output, 
execute)
+          case c => c
+        })
+      case other => other
+    }
   }
 
   private def validateStoreAssignmentPolicy(): Unit = {
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
index d07a3e9122bd..b961c5adfbcd 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
@@ -19,17 +19,26 @@ package org.apache.spark.sql.catalyst.analysis
 import scala.collection.mutable
 
 import org.apache.spark.sql.catalyst.FunctionIdentifier
-import org.apache.spark.sql.catalyst.expressions.{And, CheckInvariant, 
EqualNullSafe, Expression, V2ExpressionUtils}
-import org.apache.spark.sql.catalyst.plans.logical.{Filter, LogicalPlan, 
V2WriteCommand, WriteDelta}
+import org.apache.spark.sql.catalyst.expressions.{And, Cast, CheckInvariant, 
EqualNullSafe, Expression, V2ExpressionUtils}
+import org.apache.spark.sql.catalyst.plans.logical.{Filter, LogicalPlan, 
RowLevelWrite, V2WriteCommand, WriteDelta}
 import org.apache.spark.sql.catalyst.rules.Rule
 import org.apache.spark.sql.catalyst.trees.TreePattern.{COMMAND, 
LAMBDA_VARIABLE}
-import org.apache.spark.sql.catalyst.util.GeneratedColumn
+import org.apache.spark.sql.catalyst.util.{CharVarcharUtils, GeneratedColumn, 
StoredExpressionText}
+import org.apache.spark.sql.catalyst.util.TypeUtils.toSQLId
 import org.apache.spark.sql.connector.catalog.{CatalogManager, 
GenerationExpression}
 import org.apache.spark.sql.connector.catalog.constraints.Check
 import org.apache.spark.sql.connector.write.RowLevelOperation
+import org.apache.spark.sql.errors.QueryCompilationErrors
 import org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation
 
-class ResolveTableConstraints(val catalogManager: CatalogManager) extends 
Rule[LogicalPlan] {
+/**
+ * @param analyze Analyzes a plan under the active SQLConf. The check of a 
generated column goes
+ *                through it, so that the settings its stored text is read 
under cover its analysis
+ *                without applying to the query of the write.
+ */
+class ResolveTableConstraints(
+    val catalogManager: CatalogManager,
+    analyze: LogicalPlan => LogicalPlan) extends Rule[LogicalPlan] {
 
   override def apply(plan: LogicalPlan): LogicalPlan = 
plan.resolveOperatorsWithPruning(
     _.containsPattern(COMMAND), ruleId) {
@@ -108,6 +117,13 @@ class ResolveTableConstraints(val catalogManager: 
CatalogManager) extends Rule[L
       return Seq.empty
     }
 
+    // A DELETE writes no new generated value, so the rows it carries have 
nothing to check.
+    v2Write match {
+      case w: RowLevelWrite if w.operation.command() == 
RowLevelOperation.Command.DELETE =>
+        return Seq.empty
+      case _ =>
+    }
+
     val resolver = catalogManager.v1SessionCatalog.conf.resolver
     val autoFilledGenCols = r.output
       .filter(GeneratedColumn.isAutoFilledGeneratedColumn)
@@ -119,27 +135,52 @@ class ResolveTableConstraints(val catalogManager: 
CatalogManager) extends Rule[L
       Option(col.columnGenerationExpression())
         .filter(_ => !autoFilledGenCols.exists(n => resolver(n, col.name)))
         .map { genExpr =>
-          val catalystExpr = buildGenerationCatalystExpression(genExpr)
+          val catalystExpr = buildGenerationCatalystExpression(r, col.name, 
genExpr)
           val colRef = UnresolvedAttribute.quoted(col.name)
           val columnExtractors = Seq(col.name -> colRef)
           val genExprSql = Option(genExpr.getSql()).getOrElse(catalystExpr.sql)
-          CheckInvariant(
-            EqualNullSafe(colRef, catalystExpr),
+          // Compare with the value an auto-fill stores, which store 
assignment casts to the column
+          // type. Otherwise, the comparison can cast the provided value to 
the expression type.
+          val colType = 
CharVarcharUtils.replaceCharVarcharWithString(col.dataType())
+          def check(expr: Expression): Expression = CheckInvariant(
+            EqualNullSafe(
+              colRef, Cast(expr, colType, Option(conf.sessionLocalTimeZone), 
ansiEnabled = true)),
             columnExtractors,
             "Generated Column",
             s"${col.name} <=> $genExprSql")
+          // Analyze the check under the settings an auto-filled value is 
analyzed under, so that a
+          // row Spark auto-filled passes it when a later write carries it.
+          StoredExpressionText.analyze(
+            check(catalystExpr),
+            
check(Option(genExpr.getSql).map(StoredExpressionText.parseWithSessionAnsi)
+              .getOrElse(catalystExpr)),
+            v2Write.query.output,
+            analyze)
         }
     }.toSeq
   }
 
   /**
-   * Convert a V2 GenerationExpression to a Catalyst expression.
-   * Try the V2 expression first, fall back to parsing the SQL string.
+   * Convert a V2 GenerationExpression to a Catalyst expression. Read the SQL 
string first, as an
+   * auto-filled value does, and fall back to the V2 expression.
    */
-  private def buildGenerationCatalystExpression(genExpr: 
GenerationExpression): Expression = {
-    Option(genExpr.getExpression)
-      .flatMap(V2ExpressionUtils.toCatalyst)
-      
.getOrElse(catalogManager.v1SessionCatalog.parser.parseExpression(genExpr.getSql))
+  private def buildGenerationCatalystExpression(
+      r: DataSourceV2Relation,
+      colName: String,
+      genExpr: GenerationExpression): Expression = {
+    Option(genExpr.getSql)
+      .map(StoredExpressionText.parse)
+      
.orElse(Option(genExpr.getExpression).flatMap(V2ExpressionUtils.toCatalyst))
+      .getOrElse {
+        val operation = s"a value for the generated column 
${toSQLId(colName)}, because " +
+          s"Spark cannot convert its expression ${genExpr.getExpression}"
+        // A relation of a TableProvider has no catalog and no identifier.
+        throw ((r.catalog, r.identifier) match {
+          case (Some(catalog), Some(ident)) =>
+            QueryCompilationErrors.unsupportedTableOperationError(catalog, 
ident, operation)
+          case _ => 
QueryCompilationErrors.unsupportedTableOperationError(r.table, operation)
+        })
+      }
   }
 
   private def buildColumnExtractors(
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala
index 82f15532c542..c58d1a1dfe57 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala
@@ -119,10 +119,11 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
         AnsiStoreAssignmentCastCheck.DEFAULT): LogicalPlan = {
     resolveOutputColumnsInternal(
       tableName, expected, query, byName, conf, defaultValueFillMode,
-      ansiStoreAssignmentCastCheck)._1
+      ansiStoreAssignmentCastCheck, CatalystSqlParser.parseExpression)._1
   }
 
   /**
+   * @param readGenerationExpr reads the stored text of a generation 
expression.
    * @return a tuple of:
    *   - the resolved write plan (same as [[resolveOutputColumns]]); and
    *   - the names of the generated columns that were auto-filled, i.e. 
computed from their
@@ -135,11 +136,12 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
       byName: Boolean,
       conf: SQLConf,
       defaultValueFillMode: DefaultValueFillMode.Value = NONE,
-      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck
+      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck,
+      readGenerationExpr: String => Expression
   ): (LogicalPlan, Set[String]) = {
     resolveOutputColumnsInternal(
       tableName, expected, query, byName, conf, defaultValueFillMode,
-      ansiStoreAssignmentCastCheck)
+      ansiStoreAssignmentCastCheck, readGenerationExpr)
   }
 
   private def resolveOutputColumnsInternal(
@@ -149,7 +151,8 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
       byName: Boolean,
       conf: SQLConf,
       defaultValueFillMode: DefaultValueFillMode.Value,
-      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck
+      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck,
+      readGenerationExpr: String => Expression
   ): (LogicalPlan, Set[String]) = {
 
     if (expected.size < query.output.size) {
@@ -176,7 +179,8 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
         Nil,
         defaultValueFillMode,
         enforceFullOutput = true,
-        ansiStoreAssignmentCastCheck)
+        ansiStoreAssignmentCastCheck,
+        readGenerationExpr)
     } else {
       if (expected.size > query.output.size && !fillDefaultValue) {
         throw QueryCompilationErrors.cannotWriteNotEnoughColumnsToTableError(
@@ -185,7 +189,8 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
       resolveColumnsByPosition(
         tableName, query.output, expected, conf, errors += _,
         fillDefaultValue = fillDefaultValue,
-        ansiStoreAssignmentCastCheck = ansiStoreAssignmentCastCheck)
+        ansiStoreAssignmentCastCheck = ansiStoreAssignmentCastCheck,
+        readGenerationExpr = readGenerationExpr)
     }
 
     if (errors.nonEmpty) {
@@ -415,7 +420,8 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
       colPath: Seq[String] = Nil,
       defaultValueFillMode: DefaultValueFillMode.Value,
       enforceFullOutput: Boolean = false,
-      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck
+      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck,
+      readGenerationExpr: String => Expression = 
CatalystSqlParser.parseExpression
   ): (Seq[NamedExpression], Set[String]) = {
     // Names of generated columns that were auto-filled (not provided by the 
user). Only populated
     // for top-level columns, since generated columns cannot be nested.
@@ -433,7 +439,7 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
             // The references are left unresolved here. 
`resolveOutputColumnsInternal` resolves them
             // against the stored columns, so the generated value is computed 
from the values as
             // they will be written. References to other generated columns are 
not allowed.
-            val genExpr = CatalystSqlParser.parseExpression(genExprSql)
+            val genExpr = readGenerationExpr(genExprSql)
             Some(applyColumnMetadata(genExpr, expectedCol))
           case None =>
             val useNullAsDefault = 
useNullAsDefaultForMissingColumn(expectedCols, conf)
@@ -518,7 +524,8 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
       addError: String => Unit,
       colPath: Seq[String] = Nil,
       fillDefaultValue: Boolean = false,
-      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck
+      ansiStoreAssignmentCastCheck: AnsiStoreAssignmentCastCheck,
+      readGenerationExpr: String => Expression = 
CatalystSqlParser.parseExpression
   ): (Seq[NamedExpression], Set[String]) = {
     // Names of generated columns that were auto-filled (not provided by the 
user). Only populated
     // for top-level columns, since generated columns cannot be nested.
@@ -592,7 +599,7 @@ object TableOutputResolver extends SQLConfHelper with 
Logging {
             autoFilledGenCols += expectedCol.name
             // The references are resolved as in the by-name path. The stored 
columns have the
             // names of the table columns, even when the query uses different 
names.
-            val genExpr = CatalystSqlParser.parseExpression(genExprSql)
+            val genExpr = readGenerationExpr(genExprSql)
             applyColumnMetadata(genExpr, expectedCol)
           case None =>
             val useNullAsDefault = 
useNullAsDefaultForMissingColumn(actualExpectedCols, conf)
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpression.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpression.scala
index 19f1336559b2..9a9ad6d713ae 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpression.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpression.scala
@@ -20,7 +20,7 @@ package org.apache.spark.sql.catalyst.plans.logical
 import org.apache.spark.sql.AnalysisException
 import org.apache.spark.sql.catalyst.expressions.{AnalysisAwareExpression, 
AttributeReference, Cast, Expression, UnaryExpression, Unevaluable}
 import 
org.apache.spark.sql.catalyst.trees.TreePattern.{ANALYSIS_AWARE_EXPRESSION, 
PLAN_EXPRESSION, TreePattern, VARIABLE_REFERENCE}
-import org.apache.spark.sql.catalyst.util.V2ExpressionBuilder
+import org.apache.spark.sql.catalyst.util.{StoredExpressionText, 
V2ExpressionBuilder}
 import org.apache.spark.sql.connector.catalog.GenerationExpression
 import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.types.{DataType, TimeType}
@@ -59,6 +59,7 @@ case class GeneratedColumnExpression(
    * - No subquery expressions
    * - No references to session (temporary) variables
    * - No non-UTF8 binary collation
+   * - The text reads the same as it does under the settings a write reads it 
with
    */
   def validate(
       fieldName: String,
@@ -124,6 +125,16 @@ case class GeneratedColumnExpression(
       throw unsupportedExpressionError(
         "generation expression cannot contain non utf8 binary collated string 
type")
     }
+
+    // A write reads the stored text under fixed settings, so that a value 
does not depend on the
+    // session that writes it. The text has to read the same there. This also 
applies to a table
+    // that computes the values itself, so that the text means the same to any 
reader.
+    if (!StoredExpressionText.readsAsInSession(originalSQL)) {
+      throw unsupportedExpressionError(
+        "a write reads it with the default values of the parser configs, such 
as " +
+          "spark.sql.parser.escapedStringLiterals and 
spark.sql.ansi.enforceReservedKeywords, " +
+          "and this session reads it differently")
+    }
   }
 
   // Convert the generation expression to V2 GenerationExpression. The V2 
expression is built from
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StoredExpressionText.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StoredExpressionText.scala
new file mode 100644
index 000000000000..cdf189812a40
--- /dev/null
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StoredExpressionText.scala
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.util
+
+import org.apache.spark.sql.AnalysisException
+import org.apache.spark.sql.catalyst.analysis.{Analyzer, 
ExpressionWithUnresolvedIdentifier}
+import org.apache.spark.sql.catalyst.expressions.{Alias, Attribute, 
Expression, ExprId, NamedExpression}
+import org.apache.spark.sql.catalyst.parser.{CatalystSqlParser, ParseException}
+import org.apache.spark.sql.catalyst.plans.logical.{LocalRelation, 
LogicalPlan, Project}
+import org.apache.spark.sql.internal.SQLConf
+
+/**
+ * Reads the SQL text of a generation expression stored with a table, 
independently of the
+ * settings of the session that reads it.
+ */
+object StoredExpressionText {
+
+  /**
+   * The settings a stored text is read under. The session time zone, case 
sensitivity, timestamp
+   * type and decimal precision loss stay with the given session, because the 
settings of the DDL
+   * are not known here and pinning them would silently change existing tables 
of a deployment that
+   * sets them. Any other config that the parsing or the analysis reads takes 
its default value,
+   * also one that a function reads while it is built, such as
+   * `spark.sql.legacy.negativeIndexInArrayInsert`. With
+   * `spark.sql.legacy.useCurrentConfigsForGeneratedColumn`, these are the 
settings of the session.
+   */
+  def conf(session: SQLConf): SQLConf = {
+    if (session.useCurrentSQLConfigsForGeneratedColumn) {
+      session.clone()
+    } else {
+      val c = new SQLConf
+      Analyzer.retainResolutionConfigsForAnalysis(newConf = c, existingConf = 
session)
+      c.setConfString(SQLConf.ANSI_ENABLED.key, "true")
+      c.setConfString(SQLConf.SESSION_LOCAL_TIMEZONE.key, 
session.sessionLocalTimeZone)
+      c.setConfString(SQLConf.CASE_SENSITIVE.key, 
session.caseSensitiveAnalysis.toString)
+      c.setConf(SQLConf.TIMESTAMP_TYPE, 
session.getConf(SQLConf.TIMESTAMP_TYPE))
+      c.setConf(
+        SQLConf.DECIMAL_OPERATIONS_ALLOW_PREC_LOSS, 
session.decimalOperationsAllowPrecisionLoss)
+      c
+    }
+  }
+
+  /**
+   * Parses `text` under [[conf]]. A text these settings cannot parse is 
parsed under the active
+   * session, as a text they cannot resolve is analyzed under it.
+   */
+  def parse(text: String): Expression = parse(conf(SQLConf.get), text)
+
+  /**
+   * Parses `text` under [[conf]] with the ANSI value of the active session, 
for a text that
+   * [[conf]] cannot resolve and the session analyzes. Only its ANSI behavior 
follows the session.
+   */
+  def parseWithSessionAnsi(text: String): Expression = 
parse(confWithSessionAnsi(SQLConf.get), text)
+
+  private def parse(c: SQLConf, text: String): Expression = {
+    try {
+      SQLConf.withExistingConf(c) {
+        CatalystSqlParser.parseExpression(text)
+      }
+    } catch {
+      case _: ParseException => CatalystSqlParser.parseExpression(text)
+    }
+  }
+
+  private def confWithSessionAnsi(session: SQLConf): SQLConf = {
+    val c = conf(session)
+    c.setConfString(SQLConf.ANSI_ENABLED.key, session.ansiEnabled.toString)
+    c
+  }
+
+  /**
+   * Analyzes `expr` over `input` with `analyzer` under [[conf]], so that an 
auto-filled value and
+   * the check of a provided value read a stored text the same way. If these 
settings cannot
+   * analyze or resolve it, such as an identifier string that only the session 
can parse, or it
+   * analyzes to another plan, such as an aggregate that a catalog stores, 
this returns `inSession`
+   * instead, and the analysis of the write resolves that under the session.
+   */
+  def analyze[E <: Expression](
+      expr: E,
+      inSession: => E,
+      input: Seq[Attribute],
+      analyzer: LogicalPlan => LogicalPlan): E = {
+    val named = expr match {
+      case n: NamedExpression => n
+      case e => Alias(e, "value")()
+    }
+    val analyzed = try {
+      SQLConf.withExistingConf(conf(SQLConf.get)) {
+        analyzer(Project(Seq(named), LocalRelation(input)))
+      } match {
+        case Project(Seq(a), _: LocalRelation) if a.resolved => Some(a)
+        case _ => None
+      }
+    } catch {
+      case _: AnalysisException => None
+    }
+    analyzed.map {
+      case Alias(child, _) if !expr.isInstanceOf[NamedExpression] => child
+      case a => a
+    }.getOrElse(inSession).asInstanceOf[E]
+  }
+
+  /**
+   * Whether `text` reads the same under [[conf]] as it does in the active 
session. The comparison
+   * uses the ANSI value of the session, so that it is about how the text 
reads.
+   */
+  def readsAsInSession(text: String): Boolean = {
+    val fixed = confWithSessionAnsi(SQLConf.get)
+    val parsedInSession = CatalystSqlParser.parseExpression(text)
+    try {
+      val parsed = SQLConf.withExistingConf(fixed) {
+        CatalystSqlParser.parseExpression(text)
+      }
+      normalize(parsed) == normalize(parsedInSession)
+    } catch {
+      case _: ParseException => false
+    }
+  }
+
+  private def normalize(e: Expression): Expression = e.transform {
+    case a: Alias => a.withExprId(ExprId(0))
+    // Two parses of the same text never share the builder, which is a 
function.
+    case i: ExpressionWithUnresolvedIdentifier => i.copy(exprBuilder = 
noBuilder)
+  }
+
+  private val noBuilder: (Seq[String], Seq[Expression]) => Expression = (_, 
exprs) => exprs.head
+}
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
index 4af1c7d1a13d..bae1bbeea1fa 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
@@ -1260,6 +1260,10 @@ private[sql] object QueryCompilationErrors extends 
QueryErrorsBase with Compilat
     unsupportedTableOperationError(table.name(), "batch scan")
   }
 
+  def unsupportedTableOperationError(table: Table, operation: String): 
Throwable = {
+    unsupportedTableOperationError(table.name(), operation)
+  }
+
   def unsupportedBatchWriteError(table: Table): Throwable = {
     unsupportedTableOperationError(table.name(), "batch write")
   }
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
index 0ee5338a0e1c..2ad8e618f946 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
@@ -4534,6 +4534,17 @@ object SQLConf {
       .booleanConf
       .createWithDefault(false)
 
+  val USE_CURRENT_SQL_CONFIGS_FOR_GENERATED_COLUMN =
+    buildConf("spark.sql.legacy.useCurrentConfigsForGeneratedColumn")
+      .internal()
+      .doc("When true, a write parses and analyzes the SQL text of a 
generation expression of a " +
+        "Data Source V2 table under the SQL configs of the current active 
SparkSession, and " +
+        "CREATE TABLE and REPLACE TABLE do not reject a text that the session 
reads differently.")
+      .version("4.4.0")
+      .withBindingPolicy(ConfigBindingPolicy.SESSION)
+      .booleanConf
+      .createWithDefault(false)
+
   val STORE_ANALYZED_PLAN_FOR_VIEW =
     buildConf("spark.sql.legacy.storeAnalyzedPlanForView")
       .internal()
@@ -10029,6 +10040,9 @@ class SQLConf extends Serializable with Logging with 
SqlApiConf {
 
   def useCurrentSQLConfigsForView: Boolean = 
getConf(SQLConf.USE_CURRENT_SQL_CONFIGS_FOR_VIEW)
 
+  def useCurrentSQLConfigsForGeneratedColumn: Boolean =
+    getConf(SQLConf.USE_CURRENT_SQL_CONFIGS_FOR_GENERATED_COLUMN)
+
   def storeAnalyzedPlanForView: Boolean = 
getConf(SQLConf.STORE_ANALYZED_PLAN_FOR_VIEW)
 
   def allowAutoGeneratedAliasForView: Boolean = 
getConf(SQLConf.ALLOW_AUTO_GENERATED_ALIAS_FOR_VEW)
diff --git 
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpressionSuite.scala
 
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpressionSuite.scala
index de11b78452cc..1026d0aa2648 100644
--- 
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpressionSuite.scala
+++ 
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpressionSuite.scala
@@ -26,7 +26,7 @@ import org.apache.spark.sql.types.{TimestampNTZType, 
TimestampType}
 class GeneratedColumnExpressionSuite extends SparkFunSuite {
 
   private def genExpr(childType: DataType): GeneratedColumnExpression =
-    GeneratedColumnExpression(Literal.create(null, childType), "<gen expr>")
+    GeneratedColumnExpression(Literal.create(null, childType), "NULL")
 
   test("SPARK-57303: validate accepts a lossless widening to a nanosecond 
timestamp column") {
     // The generation expression's type is up-castable to the column type, so 
validate() succeeds:
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2DataFrameSuite.scala
 
b/sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2DataFrameSuite.scala
index 1fd1f84c48e1..dbbf210ab190 100644
--- 
a/sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2DataFrameSuite.scala
+++ 
b/sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2DataFrameSuite.scala
@@ -1551,7 +1551,7 @@ class DataSourceV2DataFrameSuite
   // Unlike the disabled case above, this behavior is not parameterized over 
ANSI: reserved keyword
   // enforcement (enforceReservedKeywords = ansiEnabled && 
ENFORCE_RESERVED_KEYWORDS) can only be on
   // when ANSI is enabled. With ANSI disabled the conf is a no-op.
-  test("generation expression resolves a reserved keyword to the function when 
" +
+  test("generation expression with a reserved keyword is rejected when " +
     "enforceReservedKeywords is enabled") {
     val tableName = "testcat.ns1.ns2.tbl"
     withSQLConf(
@@ -1559,21 +1559,26 @@ class DataSourceV2DataFrameSuite
         SQLConf.ENFORCE_RESERVED_KEYWORDS.key -> "true") {
       withTable(tableName) {
         // With reserved keyword enforcement on, the unquoted `current_date` 
in the generation
-        // expression binds to the built-in CURRENT_DATE function rather than 
the same-named
-        // column. This matches the SQL standard.
-        val createExec = executeAndKeepPhysicalPlan[CreateTableExec] {
-          sql(
-            s"""CREATE TABLE $tableName (
-               |  `current_date` DATE,
-               |  c2 DATE GENERATED ALWAYS AS (current_date)
-               |) USING foo""".stripMargin)
-        }
-        val genExpr = createExec.columns.find(_.name == 
"c2").get.columnGenerationExpression()
-        assert(genExpr != null)
-        assert(genExpr.getSql == "current_date")
-        // CURRENT_DATE has no V2 representation, so the connector falls back 
to the SQL form. A
-        // non-null field reference here would mean the column incorrectly 
took priority.
-        assert(genExpr.getExpression == null)
+        // expression binds to the built-in CURRENT_DATE function. A write 
reads the stored text
+        // without the enforcement and would bind it to the same-named column 
instead.
+        checkError(
+          exception = intercept[AnalysisException] {
+            sql(
+              s"""CREATE TABLE $tableName (
+                 |  `current_date` DATE,
+                 |  c2 DATE GENERATED ALWAYS AS (current_date)
+                 |) USING foo""".stripMargin)
+          },
+          condition = "UNSUPPORTED_EXPRESSION_GENERATED_COLUMN",
+          parameters = Map(
+            "fieldName" -> "c2",
+            "expressionStr" -> "current_date",
+            "reason" -> ("a write reads it with the default values of the 
parser configs, " +
+              "such as spark.sql.parser.escapedStringLiterals and " +
+              "spark.sql.ansi.enforceReservedKeywords, and this session reads 
it differently")))
+
+        // CURDATE is not a reserved keyword, so it reads as the function in 
both cases.
+        sql(s"CREATE TABLE $tableName (c2 DATE GENERATED ALWAYS AS 
(curdate())) USING foo")
       }
     }
   }
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/connector/GeneratedColumnWriteSuite.scala
 
b/sql/core/src/test/scala/org/apache/spark/sql/connector/GeneratedColumnWriteSuite.scala
index accba28d42da..525b02bfb98f 100644
--- 
a/sql/core/src/test/scala/org/apache/spark/sql/connector/GeneratedColumnWriteSuite.scala
+++ 
b/sql/core/src/test/scala/org/apache/spark/sql/connector/GeneratedColumnWriteSuite.scala
@@ -17,15 +17,19 @@
 
 package org.apache.spark.sql.connector
 
-import org.apache.spark.SparkRuntimeException
+import java.util.Collections
+
+import org.apache.spark.{SparkRuntimeException, SparkThrowable}
 import org.apache.spark.sql.{AnalysisException, DataFrame, QueryTest, Row}
 import org.apache.spark.sql.catalyst.QueryPlanningTracker
 import org.apache.spark.sql.catalyst.expressions.CheckInvariant
 import org.apache.spark.sql.catalyst.util.GeneratedColumn
-import org.apache.spark.sql.connector.catalog.{Identifier, 
InMemoryRowLevelOperationTableCatalog}
+import org.apache.spark.sql.connector.catalog.{Column, GenerationExpression, 
Identifier, InMemoryCatalog, InMemoryRowLevelOperationTableCatalog}
 import org.apache.spark.sql.connector.catalog.CatalogV2Implicits.CatalogHelper
+import org.apache.spark.sql.connector.expressions.{Expression => V2Expression, 
GeneralScalarExpression, Transform}
 import org.apache.spark.sql.execution.streaming.runtime.MemoryStream
 import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.types.{IntegerType, MetadataBuilder, StringType, 
StructType}
 
 /**
  * Tests for generated column auto-fill and constraint enforcement during 
writes.
@@ -110,6 +114,15 @@ class GeneratedColumnWriteSuite extends QueryTest with 
DatasourceV2SQLBase {
       s"generation expression leaked into the schema of $column: 
${metadata.json}")
   }
 
+  private def assertWriteFails(condition: String)(write: => Unit): Unit = {
+    val ex = intercept[SparkThrowable](write)
+    assert(ex.getCondition == condition)
+  }
+
+  private val readsDifferentlyReason = "a write reads it with the default 
values of the parser " +
+    "configs, such as spark.sql.parser.escapedStringLiterals and " +
+    "spark.sql.ansi.enforceReservedKeywords, and this session reads it 
differently"
+
   testGeneratedColumnWrite("append_data_v2") { table =>
     import testImplicits._
     Seq(Tuple5(1L, "foo", "2020-10-11 12:30:30", 100, "2020-11-12"))
@@ -1351,4 +1364,559 @@ class GeneratedColumnWriteSuite extends QueryTest with 
DatasourceV2SQLBase {
         "No CheckInvariant should be added when the table lacks the write 
capability")
     }
   }
+
+  test("SPARK-60757: an auto-filled value does not depend on the parser 
settings of the writer") {
+    import testImplicits._
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a STRING,
+             |  b STRING GENERATED ALWAYS AS (concat(a, 'x\\\\y'))
+             |) USING foo""".stripMargin)
+      sql(s"INSERT INTO testcat.$tblName(a) VALUES ('default')")
+      withSQLConf(SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES ('insert')")
+        Seq("append").toDF("a").writeTo(s"testcat.$tblName").append()
+      }
+      checkAnswer(
+        spark.table(s"testcat.$tblName"),
+        Seq("default", "insert", "append").map(a => Row(a, a + "x\\y")))
+
+      withSQLConf(SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+        sql(s"INSERT OVERWRITE testcat.$tblName(a) VALUES ('overwrite')")
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Row("overwrite", 
"overwritex\\y"))
+    }
+  }
+
+  test("SPARK-60757: the validation of a provided value does not depend on the 
parser settings " +
+    "of the writer") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a STRING,
+             |  b STRING GENERATED ALWAYS AS (concat(a, 'x\\\\y'))
+             |) USING foo""".stripMargin)
+      withSQLConf(SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+        sql(s"INSERT INTO testcat.$tblName(a, b) SELECT 'p', concat('p', 'x', 
chr(92), 'y')")
+        assertWriteFails("CHECK_CONSTRAINT_VIOLATION") {
+          sql(s"INSERT INTO testcat.$tblName(a, b) " +
+            "SELECT 'q', concat('q', 'x', chr(92), chr(92), 'y')")
+        }
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Row("p", "px\\y"))
+    }
+  }
+
+  test("SPARK-60757: a later write carries a row that a non-ANSI writer 
auto-filled") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a STRING,
+               |  b DOUBLE GENERATED ALWAYS AS (a + 1)
+               |) USING foo""".stripMargin)
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES ('9007199254740993')")
+        sql(s"INSERT INTO testcat.$tblName SELECT * FROM testcat.$tblName")
+      }
+      val row = Row("9007199254740993", 9007199254740994D)
+      checkAnswer(spark.table(s"testcat.$tblName"), Seq(row, row))
+    }
+  }
+
+  test("SPARK-60757: a DELETE does not check the rows that it carries") {
+    withRowLevelCatalog {
+      val tblName = "my_tab"
+      withTable(s"$rowLevelCat.$tblName") {
+        withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+          // Without the capability, the rows go in as a non-ANSI writer 
stored them before.
+          sql(s"""CREATE TABLE $rowLevelCat.$tblName(
+                 |  id INT,
+                 |  a STRING,
+                 |  b DOUBLE GENERATED ALWAYS AS (a + 1)
+                 |) USING foo TBLPROPERTIES ('$noWriteCapProp' = 
'false')""".stripMargin)
+          sql(s"INSERT INTO $rowLevelCat.$tblName VALUES " +
+            "(1, 'abc', NULL), (2, '2', 3D), (3, '3', 4D)")
+          sql(s"ALTER TABLE $rowLevelCat.$tblName SET TBLPROPERTIES 
('$noWriteCapProp' = 'true')")
+          sql(s"DELETE FROM $rowLevelCat.$tblName WHERE id >= 3")
+        }
+        withSQLConf(SQLConf.ANSI_ENABLED.key -> "true") {
+          sql(s"DELETE FROM $rowLevelCat.$tblName WHERE id >= 2")
+        }
+        checkAnswer(spark.table(s"$rowLevelCat.$tblName"), Row(1, "abc", null))
+        assertWriteFails("CAST_INVALID_INPUT") {
+          sql(s"INSERT INTO $rowLevelCat.$tblName SELECT * FROM 
$rowLevelCat.$tblName")
+        }
+      }
+    }
+  }
+
+  test("SPARK-60757: an explicit cast and arithmetic keep failing under a 
non-ANSI writer") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a STRING,
+             |  b INT GENERATED ALWAYS AS (CAST(a AS INT)),
+             |  c INT,
+             |  d INT GENERATED ALWAYS AS (c + 1)
+             |) USING foo""".stripMargin)
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        assertWriteFails("CAST_INVALID_INPUT") {
+          sql(s"INSERT INTO testcat.$tblName(a, c) VALUES ('abc', 1)")
+        }
+        assertWriteFails("ARITHMETIC_OVERFLOW") {
+          sql(s"INSERT INTO testcat.$tblName(a, c) VALUES ('1', 
${Int.MaxValue})")
+        }
+        sql(s"INSERT INTO testcat.$tblName(a, c) VALUES ('1', 1)")
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Row("1", 1, 1, 2))
+    }
+  }
+
+  test("SPARK-60757: the validation of a provided value keeps failing under a 
non-ANSI writer") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      // UPPER has no way back from its V2 form, so the validation reads the 
SQL text.
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a STRING,
+             |  b INT GENERATED ALWAYS AS (CAST(upper(a) AS INT))
+             |) USING foo""".stripMargin)
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        assertWriteFails("CAST_INVALID_INPUT") {
+          sql(s"INSERT INTO testcat.$tblName(a, b) VALUES ('abc', NULL)")
+        }
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Nil)
+    }
+  }
+
+  test("SPARK-60757: a reserved keyword of an auto-filled column does not 
follow the settings " +
+    "of the writer") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  `current_date` DATE,
+             |  b DATE GENERATED ALWAYS AS (current_date)
+             |) USING foo""".stripMargin)
+      withSQLConf(
+          SQLConf.ANSI_ENABLED.key -> "true",
+          SQLConf.ENFORCE_RESERVED_KEYWORDS.key -> "true") {
+        sql(s"INSERT INTO testcat.$tblName(`current_date`) VALUES 
(DATE'2024-06-15')")
+      }
+      val date = sqlDate("2024-06-15")
+      checkAnswer(spark.table(s"testcat.$tblName"), Row(date, date))
+    }
+  }
+
+  test("SPARK-60757: an implicit cast of an auto-filled column does not follow 
a non-ANSI writer") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "true") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a STRING,
+               |  b BIGINT GENERATED ALWAYS AS (a + 1)
+               |) USING foo""".stripMargin)
+      }
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        assertWriteFails("CAST_INVALID_INPUT") {
+          sql(s"INSERT INTO testcat.$tblName(a) VALUES ('abc')")
+        }
+        // A non-ANSI cast to DOUBLE would lose the last digit.
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES ('9007199254740993')")
+      }
+      checkAnswer(
+        spark.table(s"testcat.$tblName"),
+        Row("9007199254740993", 9007199254740994L))
+    }
+  }
+
+  test("SPARK-60757: a function of an auto-filled column does not follow the 
settings of the " +
+    "writer") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a INT,
+             |  arr ARRAY<INT>,
+             |  b INT GENERATED ALWAYS AS (abs(a)),
+             |  c INT GENERATED ALWAYS AS (element_at(arr, 2)),
+             |  d INT GENERATED ALWAYS AS (size(arr))
+             |) USING foo""".stripMargin)
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        assertWriteFails("ARITHMETIC_OVERFLOW") {
+          sql(s"INSERT INTO testcat.$tblName(a, arr) VALUES (${Int.MinValue}, 
array(1, 2))")
+        }
+        assertWriteFails("INVALID_ARRAY_INDEX_IN_ELEMENT_AT") {
+          sql(s"INSERT INTO testcat.$tblName(a, arr) VALUES (1, array(1))")
+        }
+      }
+      withSQLConf(
+          SQLConf.ANSI_ENABLED.key -> "false",
+          SQLConf.LEGACY_SIZE_OF_NULL.key -> "true") {
+        sql(s"INSERT INTO testcat.$tblName(a, arr) VALUES (1, NULL)")
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Row(1, null, 1, null, 
null))
+    }
+  }
+
+  test("SPARK-60757: a temporary function does not replace a function of an 
auto-filled column") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a STRING,
+             |  b STRING GENERATED ALWAYS AS (upper(a))
+             |) USING foo""".stripMargin)
+      spark.udf.register("upper", (s: String) => s + "-temp")
+      try {
+        withSQLConf(SQLConf.SESSION_FUNCTION_RESOLUTION_ORDER.key -> "first") {
+          sql(s"INSERT INTO testcat.$tblName(a) VALUES ('x')")
+        }
+      } finally {
+        sql("DROP TEMPORARY FUNCTION IF EXISTS upper")
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Row("x", "X"))
+    }
+  }
+
+  test("SPARK-60757: an IDENTIFIER clause is not a different reading of a 
generation expression") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a INT,
+             |  b INT GENERATED ALWAYS AS (IDENTIFIER('a') + 1)
+             |) USING foo""".stripMargin)
+      sql(s"INSERT INTO testcat.$tblName(a) VALUES (1)")
+      checkAnswer(spark.table(s"testcat.$tblName"), Row(1, 2))
+    }
+  }
+
+  test("SPARK-60757: CREATE and REPLACE TABLE reject a generation expression 
the session reads " +
+    "differently") {
+    val tblName = "my_tab"
+    for {
+      (confs, genExpr) <- Seq(
+        (Seq(SQLConf.ESCAPED_STRING_LITERALS.key -> "true"), "concat(a, 
'x\\\\y')"),
+        (Seq(SQLConf.ANSI_ENABLED.key -> "true", 
SQLConf.DOUBLE_QUOTED_IDENTIFIERS.key -> "true"),
+          "concat(\"a\", 'x')"))
+      create <- Seq("CREATE TABLE", "CREATE OR REPLACE TABLE")
+    } withSQLConf(confs: _*) {
+      checkError(
+        exception = intercept[AnalysisException] {
+          sql(s"""$create testcat.$tblName(
+                 |  a STRING,
+                 |  b STRING GENERATED ALWAYS AS ($genExpr)
+                 |) USING foo""".stripMargin)
+        },
+        condition = "UNSUPPORTED_EXPRESSION_GENERATED_COLUMN",
+        parameters = Map(
+          "fieldName" -> "b",
+          "expressionStr" -> genExpr,
+          "reason" -> readsDifferentlyReason))
+    }
+  }
+
+  test("SPARK-60757: createTable with a schema rejects a generation expression 
the session " +
+    "reads differently") {
+    val metadata = new MetadataBuilder()
+      .putString(GeneratedColumn.GENERATION_EXPRESSION_METADATA_KEY, 
"concat(a, 'x\\\\y')")
+      .build()
+    val schema = new StructType()
+      .add("a", StringType)
+      .add("b", StringType, nullable = true, metadata)
+    val tblName = "my_tab"
+    withSQLConf(SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+      checkError(
+        exception = intercept[AnalysisException] {
+          spark.catalog.createTable(s"testcat.$tblName", "foo", schema, 
Map.empty[String, String])
+        },
+        condition = "UNSUPPORTED_EXPRESSION_GENERATED_COLUMN",
+        parameters = Map(
+          "fieldName" -> "b",
+          "expressionStr" -> "concat(a, 'x\\\\y')",
+          "reason" -> readsDifferentlyReason))
+    }
+  }
+
+  test("SPARK-60757: a stored text that is not a scalar expression fails the 
analysis of the " +
+    "write") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      // A catalog can store a text that CREATE TABLE rejects.
+      catalog("testcat").asInstanceOf[InMemoryCatalog].createTable(
+        Identifier.of(Array(), tblName),
+        Array(
+          Column.create("i", IntegerType),
+          Column.create("b", IntegerType, true, null, "sum(i)", null)),
+        Array.empty[Transform],
+        Collections.emptyMap[String, String])
+      assertWriteFails("MISSING_GROUP_BY") {
+        sql(s"INSERT INTO testcat.$tblName(i) VALUES (1)")
+      }
+    }
+  }
+
+  test("SPARK-60757: a parser setting of the session does not reject an 
expression that reads " +
+    "the same") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a STRING,
+               |  b STRING GENERATED ALWAYS AS (concat(a, 'xy'))
+               |) USING foo""".stripMargin)
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES ('v')")
+      }
+      checkAnswer(spark.table(s"testcat.$tblName"), Row("v", "vxy"))
+    }
+  }
+
+  test("SPARK-60757: the validation of a provided value reads the text of a 
table with a V2 form") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        // The V2 form is COALESCE(s, '1'), with the implicit cast of a 
non-ANSI session.
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  s STRING,
+               |  b STRING GENERATED ALWAYS AS (coalesce(s, 1))
+               |) USING foo""".stripMargin)
+        val table = 
catalog("testcat").asTableCatalog.loadTable(Identifier.of(Array(), tblName))
+        val genExpr = table.columns().find(_.name == 
"b").get.columnGenerationExpression()
+        assert(genExpr.getExpression != null)
+        sql(s"INSERT INTO testcat.$tblName VALUES ('007', '7')")
+        sql(s"INSERT INTO testcat.$tblName(s) VALUES ('008')")
+        checkAnswer(spark.table(s"testcat.$tblName"), Seq(Row("007", "7"), 
Row("008", "8")))
+      }
+    }
+  }
+
+  test("SPARK-60757: the validation of a provided value does not substitute a 
variable") {
+    import testImplicits._
+    val tblName = "my_tab"
+    val variable = s"$${system:java.version}"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.VARIABLE_SUBSTITUTE_ENABLED.key -> "false") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a STRING,
+               |  b STRING GENERATED ALWAYS AS (concat(a, '$variable'))
+               |) USING foo""".stripMargin)
+        sql(s"INSERT INTO testcat.$tblName VALUES ('w', 'w$variable')")
+      }
+      // Substitution is on by default. A DataFrame write keeps the variable 
in its values.
+      Seq(("v", s"v$variable")).toDF("a", 
"b").writeTo(s"testcat.$tblName").append()
+      checkAnswer(
+        spark.table(s"testcat.$tblName"),
+        Seq(Row("w", s"w$variable"), Row("v", s"v$variable")))
+    }
+  }
+
+  test("SPARK-60757: a text the fixed settings cannot resolve is read under 
the session") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  d DATE,
+               |  s STRING,
+               |  b BOOLEAN GENERATED ALWAYS AS (CAST(d AS BOOLEAN)),
+               |  c INT GENERATED ALWAYS AS (CAST(s AS INT))
+               |) USING foo""".stripMargin)
+        sql(s"INSERT INTO testcat.$tblName(d, s) VALUES (DATE'2024-01-01', 
'1')")
+        sql(s"INSERT INTO testcat.$tblName VALUES (DATE'2024-01-02', '2', 
NULL, 2)")
+        checkAnswer(
+          spark.table(s"testcat.$tblName"),
+          Seq(
+            Row(sqlDate("2024-01-01"), "1", null, 1),
+            Row(sqlDate("2024-01-02"), "2", null, 2)))
+        // The other column of the same write stays with the fixed settings.
+        assertWriteFails("CAST_INVALID_INPUT") {
+          sql(s"INSERT INTO testcat.$tblName(d, s) VALUES (DATE'2024-01-03', 
'abc')")
+        }
+      }
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "true") {
+        assertWriteFails("DATATYPE_MISMATCH.CAST_WITH_CONF_SUGGESTION") {
+          sql(s"INSERT INTO testcat.$tblName(d, s) VALUES (DATE'2024-01-04', 
'4')")
+        }
+      }
+    }
+  }
+
+  test("SPARK-60757: a text the fixed settings cannot parse is read under the 
session") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      // A table created before the DDL rejected this text.
+      catalog("testcat").asInstanceOf[InMemoryCatalog].createTable(
+        Identifier.of(Array(), tblName),
+        Array(
+          Column.create("s", new StructType().add("f", IntegerType)),
+          Column.create("b", IntegerType, true, null, "s.\"f\"", null)),
+        Array.empty[Transform],
+        Collections.emptyMap[String, String])
+      val autoFilled = s"INSERT INTO testcat.$tblName(s) VALUES 
(named_struct('f', 1))"
+      val provided = s"INSERT INTO testcat.$tblName VALUES (named_struct('f', 
1), 1)"
+      withSQLConf(
+          SQLConf.ANSI_ENABLED.key -> "true",
+          SQLConf.DOUBLE_QUOTED_IDENTIFIERS.key -> "true") {
+        assert(!hasCheckInvariant(autoFilled))
+        assert(hasCheckInvariant(provided))
+      }
+      withSQLConf(SQLConf.DOUBLE_QUOTED_IDENTIFIERS.key -> "false") {
+        assertWriteFails("PARSE_SYNTAX_ERROR")(hasCheckInvariant(autoFilled))
+      }
+    }
+  }
+
+  test("SPARK-60757: a text the fixed settings cannot resolve keeps their 
parser settings") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.ANSI_ENABLED.key -> "false") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  d DATE,
+               |  s STRING,
+               |  b BOOLEAN GENERATED ALWAYS AS (CAST(d AS BOOLEAN) OR s = 
'x\\\\y')
+               |) USING foo""".stripMargin)
+        withSQLConf(SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+          sql(s"INSERT INTO testcat.$tblName(d, s) " +
+            "SELECT DATE'2024-01-01', concat('x', chr(92), 'y')")
+          sql(s"INSERT INTO testcat.$tblName " +
+            "SELECT DATE'2024-01-02', concat('x', chr(92), 'y'), true")
+        }
+        checkAnswer(
+          spark.table(s"testcat.$tblName"),
+          Seq(
+            Row(sqlDate("2024-01-01"), "x\\y", true),
+            Row(sqlDate("2024-01-02"), "x\\y", true)))
+      }
+    }
+  }
+
+  test("SPARK-60757: a text the fixed settings cannot analyze is read under 
the session") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(
+          SQLConf.ANSI_ENABLED.key -> "true",
+          SQLConf.DOUBLE_QUOTED_IDENTIFIERS.key -> "true") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a INT,
+               |  b INT GENERATED ALWAYS AS (IDENTIFIER('"a"') + 1)
+               |) USING foo""".stripMargin)
+        assert(!hasCheckInvariant(s"INSERT INTO testcat.$tblName(a) VALUES 
(1)"))
+        assert(hasCheckInvariant(s"INSERT INTO testcat.$tblName VALUES (5, 
6)"))
+      }
+    }
+  }
+
+  test("SPARK-60757: a V2 expression without a text that does not convert 
names its column") {
+    val tblName = "my_tab"
+    val genExpr = new GeneralScalarExpression("NOT_A_FUNCTION", 
Array.empty[V2Expression])
+    withTable(s"testcat.$tblName") {
+      catalog("testcat").asInstanceOf[InMemoryCatalog].createTable(
+        Identifier.of(Array(), tblName),
+        Array(
+          Column.create("a", IntegerType),
+          Column.builderFor("b", IntegerType)
+            .generationExpression(new GenerationExpression(genExpr)).build()),
+        Array.empty[Transform],
+        Collections.emptyMap[String, String])
+      checkError(
+        exception = intercept[AnalysisException] {
+          sql(s"INSERT INTO testcat.$tblName VALUES (1, 2)")
+        },
+        condition = "UNSUPPORTED_FEATURE.TABLE_OPERATION",
+        parameters = Map(
+          "tableName" -> "`testcat`.`my_tab`",
+          "operation" -> ("a value for the generated column `b`, because Spark 
cannot convert " +
+            s"its expression $genExpr")))
+    }
+  }
+
+  test("SPARK-60757: the validation of a provided value compares it with the 
stored value") {
+    val tblName = "my_tab"
+    Seq("false", "true").foreach { ansi =>
+      withTable(s"testcat.$tblName") {
+        withSQLConf(SQLConf.ANSI_ENABLED.key -> ansi) {
+          sql(s"""CREATE TABLE testcat.$tblName(
+                 |  s STRING,
+                 |  b STRING GENERATED ALWAYS AS (coalesce(s, 1))
+                 |) USING foo""".stripMargin)
+          sql(s"INSERT INTO testcat.$tblName VALUES ('007', '7')")
+          assertWriteFails("CHECK_CONSTRAINT_VIOLATION") {
+            sql(s"INSERT INTO testcat.$tblName VALUES ('007', '07')")
+          }
+          checkAnswer(spark.table(s"testcat.$tblName"), Row("007", "7"))
+        }
+      }
+    }
+  }
+
+  test("SPARK-60757: the check of a provided value reports the error of the 
expression") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      sql(s"""CREATE TABLE testcat.$tblName(
+             |  a INT,
+             |  c DECIMAL(20, 0) GENERATED ALWAYS AS (a div 0)
+             |) USING foo""".stripMargin)
+      assertWriteFails("DIVIDE_BY_ZERO") {
+        sql(s"INSERT INTO testcat.$tblName VALUES (1, 0)")
+      }
+    }
+  }
+
+  test("SPARK-60757: a legacy config reads a generation expression under the 
session") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(
+          SQLConf.USE_CURRENT_SQL_CONFIGS_FOR_GENERATED_COLUMN.key -> "true",
+          SQLConf.ESCAPED_STRING_LITERALS.key -> "true") {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a STRING,
+               |  b STRING GENERATED ALWAYS AS (concat(a, 'x\\\\y'))
+               |) USING foo""".stripMargin)
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES ('v')")
+        // The check of a provided value reads the text under the session too.
+        sql(s"INSERT INTO testcat.$tblName VALUES ('w', 'wx\\\\y')")
+      }
+      checkAnswer(
+        spark.table(s"testcat.$tblName"),
+        Seq(Row("v", "vx\\\\y"), Row("w", "wx\\\\y")))
+    }
+  }
+
+  test("SPARK-60757: a generation expression reads the timestamp type of the 
session") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(
+          SQLConf.SESSION_LOCAL_TIMEZONE.key -> "America/Los_Angeles",
+          SQLConf.TIMESTAMP_TYPE.key -> 
SQLConf.TimestampTypes.TIMESTAMP_NTZ.toString) {
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a STRING,
+               |  b TIMESTAMP_NTZ GENERATED ALWAYS AS (to_timestamp(a))
+               |) USING foo""".stripMargin)
+        // This time does not exist in the time zone, so a TIMESTAMP_LTZ value 
would read 03:30.
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES ('2024-03-10 02:30:00')")
+        checkAnswer(
+          spark.table(s"testcat.$tblName"),
+          Row("2024-03-10 02:30:00", 
java.time.LocalDateTime.parse("2024-03-10T02:30:00")))
+        assertWriteFails("CHECK_CONSTRAINT_VIOLATION") {
+          sql(s"INSERT INTO testcat.$tblName VALUES " +
+            "('2024-03-10 02:30:00', TIMESTAMP_NTZ'2024-03-10 03:30:00')")
+        }
+      }
+    }
+  }
+
+  test("SPARK-60757: a generation expression reads the decimal precision loss 
of the session") {
+    val tblName = "my_tab"
+    withTable(s"testcat.$tblName") {
+      withSQLConf(SQLConf.DECIMAL_OPERATIONS_ALLOW_PREC_LOSS.key -> "false") {
+        // Without the precision loss a * a is DECIMAL(38, 20), and with it 
DECIMAL(38, 17).
+        sql(s"""CREATE TABLE testcat.$tblName(
+               |  a DECIMAL(20, 10),
+               |  b DECIMAL(38, 20) GENERATED ALWAYS AS (a * a)
+               |) USING foo""".stripMargin)
+        sql(s"INSERT INTO testcat.$tblName(a) VALUES (1.1234567891)")
+        checkAnswer(
+          spark.table(s"testcat.$tblName"),
+          Row(new java.math.BigDecimal("1.1234567891"),
+            new java.math.BigDecimal("1.26215515697488187881")))
+      }
+    }
+  }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to