mkleen opened a new issue, #26965:
URL: https://github.com/apache/datafusion/issues/26965
### Describe the bug
In DataFusion, `0.0` and `-0.0` count as the same number. `GROUP BY`,
`SELECT DISTINCT` and `count(DISTINCT)` all treat them as one value, even
inside lists and structs.
`collect_set` should return each value only once, but it returns both `0.0`
and `-0.0`. It compares how the numbers are stored in memory, and the two zeros
are stored differently.
### To Reproduce
```sql
SELECT count(DISTINCT a) FROM (VALUES (0.0::double), (-0.0::double)) t(a);
-- 1
SELECT collect_set(a) FROM (VALUES (0.0::double), (-0.0::double)) t(a);
-- [-0.0, 0.0]
SELECT collect_set(a) FROM (VALUES ([0.0::double]), ([-0.0::double])) t(a);
-- [[0.0], [-0.0]]
SELECT collect_set(a) FROM (VALUES (named_struct('x', 0.0::double)),
(named_struct('x', -0.0::double))) t(a);
-- [{x: 0.0}, {x: -0.0}]
```
The same happens with `Float32`.
### Expected behavior
Each result should contain only one zero, the same way `GROUP BY` and
`count(DISTINCT)` count them:
| Input | `count(DISTINCT a)` | `collect_set(a)` today | Expected |
|---|---|---|---|
| `0.0, -0.0` | 1 | `[-0.0, 0.0]` | `[0.0]` |
| `[0.0], [-0.0]` | 1 | `[[0.0], [-0.0]]` | `[[0.0]]` |
| `{x: 0.0}, {x: -0.0}` | 1 | `[{x: 0.0}, {x: -0.0}]` | `[{x: 0.0}]` |
### Additional context
- This is one more case for #26138, which collects the places where `0.0`
and `-0.0` are still treated as different.
- Today `collect_set(DISTINCT a)` hides the problem: DataFusion first
removes duplicates the usual way, which merges the zeros. With #26753, that
step is skipped, so `collect_set(DISTINCT a)` also returns `[-0.0, 0.0]`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
