rich7420 opened a new issue, #7137: URL: https://github.com/apache/datafusion-comet/issues/7137
### Describe the bug Iceberg converts Spark's null-safe `id <=> 1` to `id = 1`. Comet pushes `NOT (id = 1)` into iceberg-rust, which evaluates it as `id != 1` and drops NULL ids. Spark keeps them, because `NOT (NULL <=> 1)` is true. The post-scan filter cannot restore them. ### Steps to reproduce ```sql CREATE TABLE t (id INT) USING iceberg; INSERT INTO t VALUES (NULL), (1), (2); SELECT * FROM t WHERE NOT (id <=> 1); ``` Reproduced at 9e1364914. ### Expected behavior `NULL` and `2`, as Spark returns. Comet returns only `2`. ### Additional context Found by @sunchao in #6847, whose partial pushdown also exposed it for mixed predicates. Spark parses `id IS DISTINCT FROM 1` and `b IS NOT TRUE` into the same negated `<=>`. #6847 stops pushing equality under `NOT` for nullable columns. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
