dongjoon-hyun opened a new pull request, #59971: URL: https://github.com/apache/spark/pull/59971
### What changes were proposed in this pull request? This PR makes `CheckInvariant`, which enforces CHECK constraints and generated columns on DSv2 writes, pass the constraint name, the predicate SQL and the column names to its generated code with `ctx.addReferenceObj`, instead of embedding them in the generated Java source as string literals. `AssertNotNull` and `PrintToStderr` pass their texts in the same way. ### Why are the changes needed? `CheckInvariant` embeds these texts in the generated Java source without escaping, so the text is compiled as Java source instead of being treated as data. - A double quote, a newline or an invalid escape sequence makes the generated code fail to compile. An ordinary condition such as `s <> "x"` hits this. With the default configuration Spark falls back to the interpreter, so the check and the data stay correct, but every such write disables whole-stage codegen and logs a compile failure for each partition. - A backslash is decoded by the Java compiler, so the violation message differs from the constraint. `s <> 'x\\y'` is shown as `s <> 'x\y'`. Generated column checks go through the same expression, so a generation expression hits the same failures. ### Does this PR introduce _any_ user-facing change? Yes. This is a bug fix for DSv2 tables in [v4.1.0](https://github.com/apache/spark/releases/tag/v4.1.0) (2025-12-11) and later. Other tables are not affected because `CheckInvariant` is only added to DSv2 writes. - Writes to a DSv2 table whose constraint text has such a character keep whole-stage codegen and no longer log compile failures. - The violation message shows the constraint name, the condition and the column names as they were written. ### How was this patch tested? Pass the CIs with the newly added test cases. The new test cases fail without this patch. ``` build/sbt 'catalyst/testOnly *ConstraintExpressionSuite' build/sbt 'sql/testOnly *CheckConstraintSuite *GeneratedColumnWriteSuite' ``` I also ran the same writes against `InMemoryTableCatalog` in a JVM without the test settings. Before this patch, compile failures and `Whole-stage codegen disabled` warnings were logged. After this patch, neither appears. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
