This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 4fecd73323f7 [SPARK-60777][SQL] Reject user-defined functions in DSv2 
CHECK constraints
4fecd73323f7 is described below

commit 4fecd73323f7598745276b336c1cbea9ee824257
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sat Oct 31 15:38:41 2026 -0700

    [SPARK-60777][SQL] Reject user-defined functions in DSv2 CHECK constraints
    
    ### What changes were proposed in this pull request?
    
    This PR makes the DDL reject a CHECK constraint whose condition calls a 
user-defined function, and makes a write bind the built-in functions of a 
condition it parses to `system.builtin`.
    
    CHECK constraints exist only on DSv2 tables. V1 and Hive tables already 
fail with `UNSUPPORTED_FEATURE.TABLE_OPERATION`.
    
    1. `CREATE TABLE`, `REPLACE TABLE` and `ALTER TABLE ... ADD CONSTRAINT` 
fail with the new error condition `USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT`, 
and the query context of the error points at the call. The check covers:
       - `UserDefinedExpression`: Scala, Java, Python and Hive functions, and 
V2 functions implementing `produceResult`.
       - `Invoke` and `StaticInvoke`: V2 functions implementing an `invoke` 
method or a static `invoke` method, which are not `UserDefinedExpression`.
       - `V2Aggregator`: V2 aggregate functions in `CREATE TABLE` and `REPLACE 
TABLE`. `ALTER TABLE` keeps rejecting them with `INVALID_WHERE_CONDITION`.
       - `SQLFunctionExpression`: SQL functions in `CREATE TABLE` and `REPLACE 
TABLE`. `ALTER TABLE` keeps rejecting them with `UNSUPPORTED_SQL_UDF_USAGE`.
    2. `ResolveTableConstraints` rewrites an unqualified `UnresolvedFunction` 
of a parsed condition to `system.builtin.<name>` when `<name>` is a built-in 
function.
    3. The SQL migration guide describes these changes.
    
    The check is in the DDL only, so an existing constraint that calls a 
user-defined function keeps working. The rewrite does not cover a built-in 
function called through an `IDENTIFIER` clause, such as 
`IDENTIFIER('upper')(s)`, because the parser does not produce an 
`UnresolvedFunction` for it, or a function injected with 
`SparkSessionExtensions.injectFunction`, which is not in 
`FunctionRegistry.builtin`.
    
    ### Why are the changes needed?
    
    A CHECK condition is stored as SQL text. A write parses that text again 
unless the stored V2 predicate converts back to a Catalyst expression, so 
function names are resolved in the session of the write. The `Check` Javadoc 
says that a condition cannot contain UDFs, but the DDL accepts them, and the 
enforced rule can change silently:
    
    ```sql
    -- a.ns.strlen and b.ns.strlen are different V2 functions with the same name
    USE a.ns;
    ALTER TABLE cat.ns.t ADD CONSTRAINT c CHECK (strlen(name) < 10);
    INSERT INTO cat.ns.t SELECT 1, 'abcdefghijklmnop';  -- 
CHECK_CONSTRAINT_VIOLATION
    USE b.ns;
    INSERT INTO cat.ns.t SELECT 2, 'abcdefghijklmnop';  -- the violating row is 
stored
    ```
    
    - A temporary function of the writer shadows an unqualified V2 function in 
the same way.
    - Dropping the function makes every write fail with `UNRESOLVED_ROUTINE`, 
and replacing it changes the rule silently.
    - A SQL function accepted by `CREATE TABLE` makes every later write fail 
with `Max iterations (100) reached for batch Resolution`.
    
    Built-in functions are affected too. `upper(s) <> 'X'` has no V2 form that 
converts back, so a write parses it. With `SET PATH = other_catalog.ns, 
system.builtin` or `spark.sql.functionResolution.sessionOrder=first`, the name 
binds to another function and the violating row is stored.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes.
    
    - Before, `CREATE TABLE` and `REPLACE TABLE` accepted a CHECK condition 
that calls a user-defined function, and `ALTER TABLE ... ADD CONSTRAINT` 
accepted it except for SQL and aggregate functions. After, the accepted 
conditions fail:
    
      ```
      [USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT] The check constraint 
`tmp_len(name) < 10` calls the user-defined function `tmp_len`. Check 
constraints can only call built-in functions. SQLSTATE: 42621
      ```
    
    - A write always binds a built-in function of a parsed condition to 
`system.builtin`, regardless of its SQL path and 
`spark.sql.functionResolution.sessionOrder`. Before, a violating row could be 
stored under those settings.
    
    Existing constraints that call a user-defined function are not rejected, 
and their functions are still resolved in the session of the write. Released 
Spark versions accept these conditions since Apache Spark 4.1.0.
    
    ### How was this patch tested?
    
    Pass the CIs with the newly added test cases in `CheckConstraintSuite`:
    
    - Rejection in `CREATE TABLE` and `REPLACE TABLE` for each kind of 
function, and in `ALTER TABLE` for the functions that it accepted before.
    - An existing constraint that calls a user-defined function stays enforced.
    - A built-in function is not shadowed under `SET PATH` and 
`spark.sql.functionResolution.sessionOrder=first`.
    - Built-in functions such as `exists` and `extract` stay enforced after the 
rewrite.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    Closes #59992 from dongjoon-hyun/SPARK-60777.
    
    Authored-by: Dongjoon Hyun <[email protected]>
    Signed-off-by: Dongjoon Hyun <[email protected]>
---
 .../src/main/resources/error/error-conditions.json |   6 +
 docs/sql-migration-guide.md                        |   2 +
 .../sql/catalyst/analysis/CheckAnalysis.scala      |   3 +
 .../analysis/ResolveTableConstraints.scala         |  18 ++-
 .../sql/catalyst/analysis/ResolveTableSpec.scala   |   1 +
 .../sql/catalyst/expressions/constraints.scala     |  27 +++-
 .../command/v2/CheckConstraintSuite.scala          | 172 ++++++++++++++++++++-
 7 files changed, 225 insertions(+), 4 deletions(-)

diff --git a/common/utils/src/main/resources/error/error-conditions.json 
b/common/utils/src/main/resources/error/error-conditions.json
index 5315dba509a9..1cd4360d4ee5 100644
--- a/common/utils/src/main/resources/error/error-conditions.json
+++ b/common/utils/src/main/resources/error/error-conditions.json
@@ -10196,6 +10196,12 @@
     },
     "sqlState" : "42601"
   },
+  "USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT" : {
+    "message" : [
+      "The check constraint `<checkCondition>` calls the user-defined function 
<functionName>. Check constraints can only call built-in functions."
+    ],
+    "sqlState" : "42621"
+  },
   "USER_RAISED_EXCEPTION" : {
     "message" : [
       "<errorMessage>"
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 99798a3e5001..2d4bcec04262 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -67,6 +67,8 @@ license: |
 - Since Spark 4.4, when a format is specified, `to_timestamp`, 
`try_to_timestamp`, and `to_timestamp_ltz` cast a numeric first argument to a 
string and parse it with the format, for example `to_timestamp(20201231, 
'yyyyMMdd')` returns `2020-12-31 00:00:00`. This matches the existing behavior 
when `spark.sql.timestampType` is `TIMESTAMP_NTZ`. Previously, with the default 
`TIMESTAMP_LTZ` timestamp type, such calls failed with `INTERNAL_ERROR`. Note 
that the string form of the number is par [...]
 - Since Spark 4.4, the Spark Thrift Server logs a warning at startup when 
`hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies 
user identities (any recognized type other than `NONE`/`NOSASL`), because 
executor-side data access still runs as the server's own service identity 
rather than the connecting user (SPARK-5159). Set 
`spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true` to acknowledge the 
limitation and silence the warning. Spark 5.0 is expected to [...]
 - Since Spark 4.4, `UPDATE` and `MERGE` on a Data Source V2 table fail with 
`DATATYPE_MISMATCH.INVALID_ROW_LEVEL_OPERATION_ASSIGNMENTS` when they assign to 
a metadata column of the table. This includes `UPDATE SET *` and `INSERT *` in 
`MERGE WITH SCHEMA EVOLUTION` when the source has a column with the name of a 
metadata column of the target table. Previously, these assignments passed 
analysis and were usually ignored. To restore the previous behavior, set 
`spark.sql.legacy.ignoreMetadata [...]
+- Since Spark 4.4, `CREATE TABLE`, `REPLACE TABLE`, and `ALTER TABLE ... ADD 
CONSTRAINT` on a Data Source V2 table fail with 
`USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT` when the condition of a CHECK 
constraint calls a user-defined function, including a Data Source V2 function. 
`ALTER TABLE ... ADD CONSTRAINT` keeps failing with `UNSUPPORTED_SQL_UDF_USAGE` 
on a SQL function and with `INVALID_WHERE_CONDITION` on an aggregate function. 
Previously, a write could resolve the function name in  [...]
+- Since Spark 4.4, a write to a Data Source V2 table that parses the SQL text 
of a CHECK constraint binds an unqualified built-in function name to 
`system.builtin`, regardless of the SQL path and 
`spark.sql.functionResolution.sessionOrder` (SPARK-60777). Previously, an 
existing constraint could call another function with the same name.
 
 ## Upgrading from Spark SQL 4.2 to 4.3
 
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
index e92a811f041b..1dad9d634136 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
@@ -1052,6 +1052,9 @@ trait CheckAnalysis extends LookupCatalog with 
QueryErrorsBase with PlanToString
               messageParameters = Map("checkCondition" -> 
a.checkConstraint.condition)
             )
 
+          case a: AddCheckConstraint =>
+            a.checkConstraint.failIfCallsUserDefinedFunction()
+
           case alter: AlterTableCommand =>
             checkAlterTableCommand(alter)
 
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
index 580af46533f3..303bc09125b9 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableConstraints.scala
@@ -18,6 +18,7 @@ package org.apache.spark.sql.catalyst.analysis
 
 import scala.collection.mutable
 
+import org.apache.spark.sql.catalyst.FunctionIdentifier
 import org.apache.spark.sql.catalyst.expressions.{And, CheckInvariant, 
EqualNullSafe, Expression, V2ExpressionUtils}
 import org.apache.spark.sql.catalyst.plans.logical.{Filter, LogicalPlan, 
V2WriteCommand, WriteDelta}
 import org.apache.spark.sql.catalyst.rules.Rule
@@ -76,7 +77,22 @@ class ResolveTableConstraints(val catalogManager: 
CatalogManager) extends Rule[L
   private def buildCatalystExpression(c: Check): Expression = {
     Option(c.predicate())
       .flatMap(V2ExpressionUtils.toCatalyst)
-      
.getOrElse(catalogManager.v1SessionCatalog.parser.parseExpression(c.predicateSql()))
+      .getOrElse(qualifyBuiltinFunctions(
+        
catalogManager.v1SessionCatalog.parser.parseExpression(c.predicateSql())))
+  }
+
+  /**
+   * Qualifies the built-in functions a parsed check condition calls, so that 
a temporary function
+   * or the SQL path of the session of the write cannot shadow them. Any other 
name is left as is:
+   * a user-defined function in a constraint that the DDL no longer accepts 
still resolves here.
+   * A function injected with `SparkSessionExtensions.injectFunction` is not in
+   * `FunctionRegistry.builtin`, so it is left as is too and can still be 
shadowed.
+   */
+  private def qualifyBuiltinFunctions(condition: Expression): Expression = 
condition.transform {
+    case f @ UnresolvedFunction(Seq(name), _, _, _, _, _, _, _)
+        if FunctionRegistry.builtin.functionExists(FunctionIdentifier(name)) =>
+      f.copy(nameParts =
+        Seq(CatalogManager.SYSTEM_CATALOG_NAME, 
CatalogManager.BUILTIN_NAMESPACE, name))
   }
 
   /**
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableSpec.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableSpec.scala
index 2b66ef503822..5371a87432ba 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableSpec.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableSpec.scala
@@ -95,6 +95,7 @@ object ResolveTableSpec extends Rule[LogicalPlan] {
               messageParameters = Map("checkCondition" -> check.condition)
             )
           }
+          check.failIfCallsUserDefinedFunction()
         case _ =>
       }
 
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/constraints.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/constraints.scala
index 12cfab81f959..fb910c6fc82c 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/constraints.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/constraints.scala
@@ -19,13 +19,17 @@ package org.apache.spark.sql.catalyst.expressions
 import java.util.UUID
 
 import org.apache.spark.sql.catalyst.InternalRow
-import org.apache.spark.sql.catalyst.analysis.{TypeCheckResult, 
UnresolvedException}
+import org.apache.spark.sql.catalyst.analysis.{AnalysisErrorAt, 
SQLFunctionExpression, TypeCheckResult, UnresolvedException}
+import org.apache.spark.sql.catalyst.expressions.aggregate.V2Aggregator
 import org.apache.spark.sql.catalyst.expressions.codegen.{Block, 
CodegenContext, ExprCode, FalseLiteral, TrueLiteral}
 import org.apache.spark.sql.catalyst.expressions.codegen.Block.BlockHelper
+import org.apache.spark.sql.catalyst.expressions.objects.{Invoke, StaticInvoke}
 import org.apache.spark.sql.catalyst.parser.ParseException
 import org.apache.spark.sql.catalyst.trees.CurrentOrigin
+import org.apache.spark.sql.catalyst.util.TypeUtils.toSQLId
 import org.apache.spark.sql.catalyst.util.V2ExpressionBuilder
 import org.apache.spark.sql.connector.catalog.constraints.Constraint
+import org.apache.spark.sql.connector.catalog.functions.ScalarFunction
 import org.apache.spark.sql.connector.expressions.FieldReference
 import org.apache.spark.sql.errors.QueryExecutionErrors
 import org.apache.spark.sql.types.{AbstractDataType, BooleanType, DataType}
@@ -128,6 +132,27 @@ case class CheckConstraint(
 
   override def inputTypes: Seq[AbstractDataType] = Seq(BooleanType)
 
+  /**
+   * Fails the analysis if the condition calls a user-defined function. A 
write can read `condition`
+   * again and resolve the name of the function in its own session, where the 
name can refer to
+   * another function or to none, so the constraint would not enforce what 
this DDL declares.
+   */
+  def failIfCallsUserDefinedFunction(): Unit = {
+    child.collectFirst {
+      case f: SQLFunctionExpression => (f, toSQLId(f.function.name.nameParts))
+      case f: UserDefinedExpression => (f, toSQLId(Seq(f.name)))
+      case i: StaticInvoke if i.scalarFunction.isDefined =>
+        (i, toSQLId(Seq(i.scalarFunction.get.name)))
+      case i @ Invoke(Literal(f: ScalarFunction[_], _), _, _, _, _, _, _, _) =>
+        (i, toSQLId(Seq(f.name)))
+      case a: V2Aggregator[_, _] => (a, toSQLId(Seq(a.aggrFunc.name)))
+    }.foreach { case (call, functionName) =>
+      call.failAnalysis(
+        errorClass = "USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT",
+        messageParameters = Map("checkCondition" -> condition, "functionName" 
-> functionName))
+    }
+  }
+
   def toV2Constraint: Constraint = {
     val predicate = new V2ExpressionBuilder(child, 
true).buildPredicate().orNull
     val enforced = userProvidedCharacteristic.enforced.getOrElse(true)
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/CheckConstraintSuite.scala
 
b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/CheckConstraintSuite.scala
index 52da4802ef14..9a286a14b8dd 100644
--- 
a/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/CheckConstraintSuite.scala
+++ 
b/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/CheckConstraintSuite.scala
@@ -17,15 +17,31 @@
 
 package org.apache.spark.sql.execution.command.v2
 
+import test.org.apache.spark.sql.connector.catalog.functions.JavaAverage
+import 
test.org.apache.spark.sql.connector.catalog.functions.JavaStrLen.JavaStrLenStaticMagic
+
 import org.apache.spark.SparkRuntimeException
 import org.apache.spark.sql.{AnalysisException, QueryTest, Row}
+import org.apache.spark.sql.catalyst.InternalRow
 import org.apache.spark.sql.catalyst.plans.logical.Filter
 import org.apache.spark.sql.catalyst.util.AttributeNameParser
 import org.apache.spark.sql.catalyst.util.QuotingUtils.quoteNameParts
-import org.apache.spark.sql.connector.catalog.Table
-import org.apache.spark.sql.connector.catalog.constraints.Check
+import org.apache.spark.sql.connector.{StrLen, StrLenDefault, StrLenMagic}
+import org.apache.spark.sql.connector.catalog.{Identifier, InMemoryCatalog, 
Table, TableCatalog, TableChange}
+import org.apache.spark.sql.connector.catalog.constraints.{Check, Constraint}
+import org.apache.spark.sql.connector.catalog.functions.{ScalarFunction, 
UnboundFunction}
 import org.apache.spark.sql.execution.command.DDLCommandTestUtils
 import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.types.{DataType, StringType}
+import org.apache.spark.unsafe.types.UTF8String
+
+// A V2 function that a test binds to the name of a built-in function to 
shadow it.
+private object ConstantString extends ScalarFunction[UTF8String] {
+  override def inputTypes(): Array[DataType] = Array(StringType)
+  override def resultType(): DataType = StringType
+  override def name(): String = "constant_string"
+  override def produceResult(input: InternalRow): UTF8String = 
UTF8String.fromString("OK")
+}
 
 class CheckConstraintSuite extends QueryTest with CommandSuiteBase with 
DDLCommandTestUtils {
   override protected def command: String = "Check CONSTRAINT"
@@ -170,6 +186,158 @@ class CheckConstraintSuite extends QueryTest with 
CommandSuiteBase with DDLComma
     }
   }
 
+  private def withV2Function(name: String, function: UnboundFunction)(f: => 
Unit): Unit = {
+    val funCatalog = spark.sessionState.catalogManager.catalog(s"fun_$catalog")
+      .asInstanceOf[InMemoryCatalog]
+    val ident = Identifier.of(Array("ns"), name)
+    withNamespace(s"fun_$catalog.ns") {
+      sql(s"CREATE NAMESPACE fun_$catalog.ns")
+      funCatalog.createFunction(ident, function)
+      try f finally funCatalog.dropFunction(ident)
+    }
+  }
+
+  // CREATE TABLE, REPLACE TABLE and, if `inAlter`, ALTER TABLE fail at the 
call in the condition.
+  private def checkUserDefinedFunctionRejected(
+      cond: String,
+      call: String,
+      functionName: String,
+      inAlter: Boolean = true): Unit = {
+    withNamespaceAndTable("ns", "tbl", nonPartitionCatalog) { t =>
+      sql(s"CREATE TABLE $t (id INT, name STRING) $defaultUsing")
+      val alter = if (inAlter) Seq(s"ALTER TABLE $t ADD CONSTRAINT c1 CHECK 
($cond)") else Nil
+      (alter ++ Seq(
+        s"CREATE TABLE ${t}_2 (id INT, name STRING, CONSTRAINT c1 CHECK 
($cond)) $defaultUsing",
+        s"REPLACE TABLE $t (id INT, name STRING, CONSTRAINT c1 CHECK ($cond)) 
$defaultUsing"
+      )).foreach { query =>
+        checkError(
+          exception = intercept[AnalysisException](sql(query)),
+          condition = "USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT",
+          sqlState = "42621",
+          parameters = Map("checkCondition" -> cond, "functionName" -> 
functionName),
+          context = ExpectedContext(fragment = call))
+      }
+    }
+  }
+
+  test("SPARK-60777: Check constraint cannot call a user-defined function") {
+    val strlen = s"fun_$catalog.ns.strlen"
+    Seq(
+      (StrLenDefault, s"$strlen(name) < 10", s"$strlen(name)", 
"`strlen_default`"),
+      // The optimizer folds a call with constant arguments, but the DDL does 
not.
+      (StrLenDefault, s"id < $strlen('abc')", s"$strlen('abc')", 
"`strlen_default`"),
+      (StrLenMagic, s"$strlen(name) < 10", s"$strlen(name)", "`strlen_magic`"),
+      (new JavaStrLenStaticMagic, s"$strlen(name) < 10", s"$strlen(name)", 
"`strlen`")
+    ).foreach { case (function, cond, call, functionName) =>
+      withV2Function("strlen", StrLen(function)) {
+        checkUserDefinedFunctionRejected(cond, call, functionName)
+      }
+    }
+
+    withUserDefinedFunction("tmp_len" -> true, "tmp_sql_len" -> true, 
"sql_len" -> false) {
+      spark.udf.register("tmp_len", (s: String) => s.length)
+      checkUserDefinedFunctionRejected("tmp_len(name) < 10", "tmp_len(name)", 
"`tmp_len`")
+      // ALTER TABLE keeps failing with UNSUPPORTED_SQL_UDF_USAGE on a SQL 
function.
+      sql("CREATE TEMPORARY FUNCTION tmp_sql_len(x STRING) RETURNS INT RETURN 
length(x)")
+      checkUserDefinedFunctionRejected(
+        "tmp_sql_len(name) < 10", "tmp_sql_len(name)", "`tmp_sql_len`", 
inAlter = false)
+      sql("CREATE FUNCTION sql_len(x STRING) RETURNS INT RETURN length(x)")
+      checkUserDefinedFunctionRejected(
+        "sql_len(name) < 10",
+        "sql_len(name)",
+        "`spark_catalog`.`default`.`sql_len`",
+        inAlter = false)
+    }
+  }
+
+  test("SPARK-60777: Check constraint cannot call a V2 aggregate function") {
+    withV2Function("avg", new JavaAverage) {
+      // ALTER TABLE keeps failing with the error for an aggregate in a filter.
+      val call = s"fun_$catalog.ns.avg(CAST(id AS DOUBLE))"
+      checkUserDefinedFunctionRejected(s"$call > 0", call, "`davg`", inAlter = 
false)
+    }
+  }
+
+  test("SPARK-60777: Existing check constraint calling a user-defined function 
still works") {
+    withUserDefinedFunction("tmp_len" -> true) {
+      spark.udf.register("tmp_len", (s: String) => s.length)
+      withNamespaceAndTable("ns", "tbl", nonPartitionCatalog) { t =>
+        sql(s"CREATE TABLE $t (id INT, name STRING) $defaultUsing")
+        val check = Constraint.check("c1").predicateSql("tmp_len(name) < 
10").build()
+        
spark.sessionState.catalogManager.catalog(nonPartitionCatalog).asInstanceOf[TableCatalog]
+          .alterTable(Identifier.of(Array("ns"), "tbl"), 
TableChange.addConstraint(check, null))
+        sql(s"INSERT INTO $t VALUES (1, 'abc')")
+        checkError(
+          exception = intercept[SparkRuntimeException] {
+            sql(s"INSERT INTO $t VALUES (2, 'abcdefghijklmnop')")
+          },
+          condition = "CHECK_CONSTRAINT_VIOLATION",
+          sqlState = "23001",
+          parameters = Map(
+            "constraintName" -> "c1",
+            "expression" -> "tmp_len(name) < 10",
+            "values" -> " - name : abcdefghijklmnop"))
+        checkAnswer(spark.table(t).select("id"), Row(1))
+      }
+    }
+  }
+
+  test("SPARK-60777: Check constraint is not shadowed in the session of a 
write") {
+    def checkEnforced(t: String): Unit = {
+      // The settings of the write do shadow the built-in function outside of 
the constraint.
+      checkAnswer(sql("SELECT upper('x')"), Row("OK"))
+      checkError(
+        exception = intercept[SparkRuntimeException] {
+          sql(s"INSERT INTO $t VALUES (1, 'x')")
+        },
+        condition = "CHECK_CONSTRAINT_VIOLATION",
+        sqlState = "23001",
+        parameters = Map(
+          "constraintName" -> "c1",
+          "expression" -> "UPPER(s) <> 'X'",
+          "values" -> " - s : x"))
+      checkAnswer(spark.table(t), Nil)
+    }
+
+    withV2Function("upper", StrLen(ConstantString)) {
+      withNamespaceAndTable("ns", "tbl", nonPartitionCatalog) { t =>
+        sql(s"CREATE TABLE $t (id INT, s STRING) $defaultUsing")
+        sql(s"ALTER TABLE $t ADD CONSTRAINT c1 CHECK (UPPER(s) <> 'X')")
+        withSQLConf(SQLConf.PATH_ENABLED.key -> "true") {
+          sql(s"SET PATH = fun_$catalog.ns, system.builtin")
+          try checkEnforced(t) finally sql("SET PATH = DEFAULT_PATH")
+        }
+        spark.udf.register("upper", (_: String) => "OK")
+        try {
+          withSQLConf(SQLConf.SESSION_FUNCTION_RESOLUTION_ORDER.key -> 
"first") {
+            checkEnforced(t)
+          }
+        } finally {
+          sql("DROP TEMPORARY FUNCTION upper")
+        }
+      }
+    }
+  }
+
+  test("SPARK-60777: Check constraint calling built-in functions is enforced") 
{
+    Seq(
+      "exists(a, x -> upper(CAST(x AS STRING)) = '2')",
+      "struct(id AS x).x > 0",
+      "extract(YEAR FROM d) < 2100"
+    ).foreach { cond =>
+      withNamespaceAndTable("ns", "tbl", nonPartitionCatalog) { t =>
+        sql(s"CREATE TABLE $t (id INT, a ARRAY<INT>, d DATE) $defaultUsing")
+        sql(s"ALTER TABLE $t ADD CONSTRAINT c1 CHECK ($cond)")
+        sql(s"INSERT INTO $t VALUES (1, array(2), DATE'2020-01-01')")
+        val error = intercept[SparkRuntimeException] {
+          sql(s"INSERT INTO $t VALUES (-1, array(-1), DATE'2999-01-01')")
+        }
+        assert(error.getCondition == "CHECK_CONSTRAINT_VIOLATION", cond)
+        checkAnswer(spark.table(t).select("id"), Row(1))
+      }
+    }
+  }
+
   def getConstraintCharacteristics(): Seq[(String, String)] = {
     Seq(
       ("", s"ENFORCED NORELY"),


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to