dejankrak-db opened a new pull request, #56238: URL: https://github.com/apache/spark/pull/56238
### What changes were proposed in this pull request? Two related fixes for non-foldable DEFAULT expressions (e.g. `current_user()`) on CHAR/VARCHAR columns: 1. **Fix INTERNAL_ERROR at DDL time.** `ResolveDefaultColumns.coerceDefaultValue` called `CharVarcharUtils.stringLengthCheck(ret, dataType).eval(EmptyRow)` to validate CHAR/VARCHAR default lengths at compile time, without checking foldability. For a non-foldable default such as `current_user()` (processed before it is replaced by a literal), `.eval(EmptyRow)` throws "Cannot evaluate expression", surfacing as INTERNAL_ERROR. Added an `&& ret.foldable` guard so the eager check only runs for foldable expressions. 2. **Add runtime CHAR/VARCHAR length enforcement for implicit defaults.** When a column with a DEFAULT is omitted from an INSERT column list, `TableOutputResolver.reorderColumnsByName` filled the default but did not apply `stringLengthCheck`, so oversized non-foldable defaults silently succeeded. It now wraps the default expression with the write-side length check, using `CharVarcharUtils.getRawType(metadata)` to detect CHAR/VARCHAR for both V1 and V2 tables. The explicit `DEFAULT` keyword path already goes through `checkField`. ### Why are the changes needed? `CREATE TABLE t(s CHAR(100) DEFAULT current_user()) USING parquet` (or the equivalent `ALTER TABLE ... SET DEFAULT`) previously failed with INTERNAL_ERROR. Oversized non-foldable defaults for omitted columns also bypassed length checks. ### Does this PR introduce _any_ user-facing change? Yes. Non-foldable expressions are now allowed as CHAR/VARCHAR defaults (instead of throwing INTERNAL_ERROR), and oversized non-foldable defaults on omitted INSERT columns now fail with EXCEED_LIMIT_LENGTH at runtime instead of succeeding. ### How was this patch tested? Added 6 tests to `ResolveDefaultColumnsSuite` (CREATE/ALTER with `current_user()` on CHAR/VARCHAR; foldable oversize fails at DDL; non-foldable oversize fails at INSERT for both implicit and explicit DEFAULT paths). Full suite: 19 tests pass. ### Was this patch authored or co-authored using generative AI tooling? Yes, co-authored using Claude code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
