Fly-a-Kite opened a new issue, #22490:
URL: https://github.com/apache/datafusion/issues/22490
### Describe the bug
## Summary
DataFusion `53.0.0` evaluates `-0.0 >= 0.0` as `false`.
Python and DuckDB evaluate the same comparison as `true`. This also affects
filters using `IS TRUE` / `IS NOT TRUE`.
## Environment
- `datafusion==53.0.0`
- `pyarrow==24.0.0`
- `duckdb==1.5.3`
- Python `3.12.3`
## Reproduction
```python
#!/usr/bin/env python3
from __future__ import annotations
import datafusion
import duckdb
import pyarrow as pa
from datafusion import SessionContext
def main() -> None:
python_cmp = (-0.0 >= 0.0)
duckdb_cmp = duckdb.connect(database=":memory:").execute(
"SELECT (-0.0 >= 0.0)"
).fetchone()[0]
ctx = SessionContext()
batch = pa.RecordBatch.from_pylist(
[{"id": 1, "y": 0.0}],
schema=pa.schema([
pa.field("id", pa.int64()),
pa.field("y", pa.float64(), nullable=True),
]),
)
ctx.register_record_batches("t0", [[batch]])
diagnostic = ctx.sql(
"SELECT id, y * -1 AS m_0, (y * -1) >= 0.0 AS cmp FROM t0"
).to_pandas()
filtered = ctx.sql(
"SELECT id, y * -1 AS m_0 "
"FROM t0 "
"WHERE NOT (((y * -1) >= 0.0) IS TRUE)"
).to_pandas()
print(f"datafusion={getattr(datafusion, '__version__', 'unknown')}")
print(f"pyarrow={pa.__version__}")
print(f"duckdb={duckdb.__version__}")
print(f"python comparison: {python_cmp!r}")
print(f"duckdb comparison: {duckdb_cmp!r}")
print("datafusion diagnostic:")
print(diagnostic)
print("datafusion filtered rows:")
print(filtered)
assert python_cmp is True
assert duckdb_cmp is True
assert bool(diagnostic.iloc[0]["cmp"]) is True
assert len(filtered) == 0
if __name__ == "__main__":
main()
```
## Expected behavior
`y * -1` is `-0.0`, and `-0.0 >= 0.0` should be `true`.
Therefore this filter should remove the row:
```sql
WHERE NOT (((y * -1) >= 0.0) IS TRUE)
```
Python and DuckDB agree with this expectation.
## Actual behavior
DataFusion reports the comparison as `false` and keeps the row.
Observed output:
```text
python comparison: True
duckdb comparison: True
datafusion diagnostic:
id m_0 cmp
0 1 -0.0 False
datafusion filtered rows:
id m_0
0 1 -0.0
```
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
