schenksj opened a new issue, #7480:
URL: https://github.com/apache/datafusion-comet/issues/7480
### Describe the bug
With `spark.sql.ansi.enabled=true`, the two queries below raise
`DIVIDE_BY_ZERO` in Spark, but
return rows when Comet runs them as `CometFilter` and `CometProject`. This
is the opposite of the
documented "ANSI errors from rows skipped by LIMIT" case, where Comet raises
an error that Spark
does not.
### Steps to reproduce
Run on `main` (`0f7f8e453`) with the default profile (Spark 4.1.3), in a
`CometTestBase` suite,
with no other Comet options set, over a one-file Parquet table:
```scala
sql("SELECT * FROM VALUES (1, 1), (2, 2), (0, NULL), (3, 3) AS v(x, y)")
.coalesce(1).write.parquet(path)
spark.read.parquet(path).createOrReplaceTempView("t")
```
```sql
SET spark.sql.ansi.enabled=true;
SELECT x FROM t WHERE y IS NOT NULL AND abs(10 div x) > 0;
SELECT x <> 0 AND abs(10 div x) + abs(10 div x) > 1 AS b FROM t;
```
| Query | Spark (`spark.comet.enabled=false`) | Comet | Comet plan |
| --- | --- | --- | --- |
| filter | `DIVIDE_BY_ZERO` | `[1]`, `[2]`, `[3]` | `CometProject` >
`CometFilter` |
| projection | `DIVIDE_BY_ZERO` | `[false]`, `[true]`, `[true]`, `[true]` |
`CometProject` |
### Expected behavior
Comet raises `DIVIDE_BY_ZERO` for both queries, as Spark does. If matching
Spark is not practical,
the difference is documented in the compatibility guide next to "ANSI errors
from rows skipped by
LIMIT".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
