tlfua opened a new issue, #7542: URL: https://github.com/apache/datafusion-comet/issues/7542
### Describe the bug `branch-1.2` has the ANSI bug from https://github.com/apache/datafusion-comet/issues/7246 for arithmetic and `concat`: native Comet evaluates an operand on rows where Spark skips it, so a query Spark answers fails with `CAST_INVALID_INPUT`. For example, with `spark.sql.ansi.enabled=true`: ```sql CREATE TABLE t(n int, txt string) USING parquet; INSERT INTO t VALUES (NULL, 'bad'), (3, '6'); SELECT n + CAST(txt AS INT) FROM t; -- Spark: NULL, 9. Comet: CAST_INVALID_INPUT ``` The fix on `main` is https://github.com/apache/datafusion-comet/pull/7456 (not merged yet). It cannot be cherry-picked as is: it builds on https://github.com/apache/datafusion-comet/pull/7326 (`LazyLogical`, speculative evaluation), which `branch-1.2` does not have. `branch-1.2` does have `NullShortCircuit` (https://github.com/apache/datafusion-comet/pull/6716), so a backport could wrap the operands without speculation. This issue tracks whether, and how, to backport the fix to `branch-1.2`, as `docs/source/contributor-guide/backporting.md` asks for a correctness fix. ### Steps to reproduce Run the query above on `branch-1.2` with ANSI mode on. ### Expected behavior Comet returns the same result as Spark: NULL where `n` is NULL, without evaluating `CAST(txt AS INT)` there. ### Additional context - Without speculation, the wrapped form is up to about 2x slower than eager evaluation for cheap operands such as `a * (b - c)` on batches with NULLs (benchmark in https://github.com/apache/datafusion-comet/pull/7456). Maintainers may prefer to backport with speculation included, or to accept the slowdown for correctness. - Once https://github.com/apache/datafusion-comet/pull/7456 merges, the backport would be decided per `backporting.md`. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
