PG1204 opened a new pull request, #23406: URL: https://github.com/apache/datafusion/pull/23406
## Which issue does this PR close? - Closes #15428. ## Rationale for this change `SELECT DISTINCT` on map columns used to fail with `ArrowError(NotYetImplemented)` because arrow-rs's `RowConverter` did not support Map types. That support has since landed upstream (apache/arrow-rs#7879) and is available on DataFusion main via arrow 59.1, so these queries now work. As suggested in https://github.com/apache/datafusion/issues/15428#issuecomment-4911477288 (maintainer comment), this PR adds sqllogictest coverage to lock in the behavior and close the issue. ## What changes are included in this PR? Adds a new test section to `datafusion/sqllogictest/test_files/map.slt` covering: - `SELECT DISTINCT` on a map column, including the exact reproducer from the issue (`DISTINCT ... LIMIT`), collapsing of duplicate maps and duplicate NULLs, and `DISTINCT` over map + scalar columns together - `GROUP BY` on a map column, `GROUP BY` map + scalar keys, and `HAVING` with a map grouping key - Aggregations with map inputs: `COUNT(map)`, `COUNT(DISTINCT map)`, `SUM` grouped by map, and `array_agg` of map values - Edge cases: empty maps under `DISTINCT` (equal to each other, distinct from `NULL`), NULL maps, and unsorted maps with the same entries in different order being treated as distinct values - `UNION` (distinct) on map columns - `DISTINCT` / `GROUP BY` over the existing 209-row `parquet_map.parquet` file for coverage on real Parquet-backed map data Note: a bare `MAP {'key': NULL}` literal infers a `Null` value type and fails type coercion across a VALUES list against `Map(Utf8, Int64)` rows, so the test uses `CAST(NULL AS BIGINT)` for the null map value. That coercion gap may be worth a separate follow-up issue. ## Are these changes tested? Yes, this PR is test-only. All tests pass locally via `cargo test -p datafusion-sqllogictest --test sqllogictests -- map.slt`. ## Are there any user-facing changes? No. Test-only change; no API or behavior changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
