dongjoon-hyun opened a new pull request, #60039: URL: https://github.com/apache/spark/pull/60039
### What changes were proposed in this pull request? This PR makes `CREATE TABLE` and `REPLACE TABLE` on a Data Source V2 table reject a CHECK constraint whose condition calls a generator function, such as `explode`, `posexplode`, and `inline`. They fail with the new error condition `GENERATOR_IN_CHECK_CONSTRAINT`, and the query context of the error points at the call. 1. `ResolveTableSpec` looks for a `Generator` in the resolved condition. It does this after the check for aggregate and window functions and before `CheckConstraint.failIfCallsUserDefinedFunction`, so a Hive UDTF fails like a built-in generator function. 2. The SQL migration guide describes this change. `ALTER TABLE ... ADD CONSTRAINT` already rejects these conditions, and this PR does not change its error. The check is in the DDL only, so a write to a table with an existing constraint fails as before. The error of `ALTER TABLE` is not reused because it shows the internal plan instead of the condition: ``` [UNSUPPORTED_GENERATOR.OUTSIDE_SELECT] The generator is not supported: outside the SELECT clause, found: Filter NOT isnotnull(explode(array(id#12))). SQLSTATE: 42K0E ``` `AGGREGATE_OR_WINDOW_FUNCTION_IN_CHECK_CONSTRAINT` is not reused either, because an aggregate or window function reads multiple rows, while a generator function returns multiple rows. ### Why are the changes needed? `CREATE TABLE` and `REPLACE TABLE` accept a generator function in a CHECK constraint, but no write to the table can succeed after that: ```sql CREATE TABLE cat.ns.t (id INT, CONSTRAINT c1 CHECK (explode(array(id)) IS NOT NULL)) USING foo; -- succeeds INSERT INTO cat.ns.t VALUES (1); -- UNSUPPORTED_GENERATOR.OUTSIDE_SELECT ``` https://github.com/apache/spark/pull/60005 rejected aggregate and window functions in these statements, but not generator functions. ### Does this PR introduce _any_ user-facing change? Yes. - Before, `CREATE TABLE` and `REPLACE TABLE` accepted these conditions, and every later write to the table failed. After, the statements fail: ``` [GENERATOR_IN_CHECK_CONSTRAINT] The check constraint `explode(array(id)) IS NOT NULL` calls a generator function. Check constraints compute one value for each row and cannot call functions that return multiple rows. SQLSTATE: 42621 ``` - Compared to the unreleased master branch, a Hive UDTF in `CREATE TABLE` and `REPLACE TABLE` fails with this error instead of `USER_DEFINED_FUNCTION_IN_CHECK_CONSTRAINT`. - `CREATE TABLE` on a V1 table with such a condition fails with this error instead of `UNSUPPORTED_FEATURE.TABLE_OPERATION`. Existing constraints that call a generator function are not rejected, and a write to their tables keeps failing with the same error. Released Spark versions accept these conditions since Apache Spark 4.1.0. ### How was this patch tested? Pass the CIs with the newly added test case in `CheckConstraintSuite`. It covers `CREATE TABLE` and `REPLACE TABLE` for `explode`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
