dajiaohuang opened a new pull request, #59809: URL: https://github.com/apache/spark/pull/59809
### What changes were proposed in this pull request? Keep a nested `IS NOT NULL` check on an expression after an earlier filter conjunct when the check appears later in the condition. Apply the same position guard in both `generatePredicateCode` and the CSE branch of `FilterExec.doConsume`, and add a regression test that exercises CSE enabled and disabled. Clarify the corresponding remaining exception in the SQL migration guide. ### Why are the changes needed? With ANSI mode and whole-stage codegen, `WHERE coalesce(b, cast(s AS INT)) < 0 AND cast(s AS INT) IS NOT NULL` currently evaluates the later cast check before the first conjunct. For a synthetic row with `b = 1` and `s = 'x'`, the first conjunct is FALSE, but codegen raises `CAST_INVALID_INPUT`; with whole-stage codegen disabled, the query correctly returns no rows. The generated evaluation should preserve the condition order, as `And.eval` does. ### Does this PR introduce _any_ user-facing change? Yes. Whole-stage codegen no longer raises an error from this later expression check after an earlier conjunct has already evaluated to FALSE; it matches the result when whole-stage codegen is disabled. No configuration or API changes. ### How was this patch tested? - Added and ran `sql/testOnly org.apache.spark.sql.execution.WholeStageCodegenSuite -- -z SPARK-60377`; passed 1/1 with FilterExec CSE enabled and disabled. - Ran `sql/testOnly org.apache.spark.sql.execution.WholeStageCodegenSuite -- -z SPARK-51356`; passed 3/3. - Ran `sql/testOnly org.apache.spark.sql.execution.WholeStageCodegenSuite -- -z follows`; passed 1/1 (the related SPARK-60315 expression-check ordering case). - `git diff --check` passed. Tests were run locally with the task-local JDK 21. The full Spark test suite was not run; CI was not queried. I affirm that this is an original contribution and license it to Apache Spark under the Apache License 2.0. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (gpt-6-luna) via Pi. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
