Yang Jie created SPARK-60550:
--------------------------------
Summary: CHECK constraints added by CREATE TABLE resolve
IDENTIFIER clauses and user-defined functions in the writer's session
Key: SPARK-60550
URL: https://issues.apache.org/jira/browse/SPARK-60550
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Yang Jie
CREATE TABLE converts a CHECK constraint before analysis finishes and stores it
without a predicate when the condition has no V2 form at that point, for
example because it calls current_timestamp() or a Scala UDF. Each write then
parses predicateSql in its own session (ResolveTableConstraints), so names the
CREATE TABLE resolved are resolved again:
{noformat}
DECLARE VARIABLE c STRING DEFAULT 'ts';
CREATE TABLE t (id INT, ts TIMESTAMP,
CONSTRAINT c1 CHECK (IDENTIFIER(c) <= current_timestamp() + INTERVAL 1 DAY));
DROP TEMPORARY VARIABLE c;
INSERT INTO t VALUES (1, current_timestamp()); -- UNRESOLVED_COLUMN `c`
-- spark.udf.register("plus1", (i: Int) => i + 1)
CREATE TABLE t2 (id INT, CONSTRAINT c1 CHECK (id < plus1(10)));
-- spark.udf.register("plus1", (i: Int) => i - 100)
INSERT INTO t2 VALUES (5); -- CHECK_CONSTRAINT_VIOLATION, checked with the new
plus1
{noformat}
Session variables referenced directly are covered by SPARK-57379. For ALTER
TABLE, SPARK-60032 keeps the predicate built from the optimized plan for such
conditions. Generated columns accept only built-in functions; CHECK constraints
could do the same.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
