edubraqd opened a new pull request, #24948:
URL: https://github.com/apache/datafusion/pull/24948
## Which issue does this PR close?
- Part of #24915 (the comparator half of the `Struct` / `Map` follow-up
discussed in #24916).
## Rationale for this change
`ScalarValue::partial_cmp` for structs flattened the struct into leaf
columns and compared them with the `lt` / `eq` kernels. Those reject nested
columns, so a struct with a list field was "Uncomparable", and they return NULL
for a NULL field, which the loop then skipped, so `struct(1, NULL)` and
`struct(1, 2)` compared equal (`min` and `max` both returned `{c0: 1, c1:
NULL}`). `partial_cmp_map` iterated `0..m1.len()`, the row count, as a column
index and therefore only compared the keys.
## What changes are included in this PR?
Both comparisons go through a shared `partial_cmp_rows` helper built on
`arrow::array::make_comparator`, which supports nested types and gives NULL
fields the same "sorts last" treatment as NULL list elements (matching #24938
for lists, and PostgreSQL / DuckDB for composite values). The `flatten` helper
is removed.
One existing expectation changes: `test_min_max_with_nulls` in
`min_max_struct.rs` fed `{1, "a"}, {NULL, NULL}, {3, "c"}` and expected `max =
{3, "c"}` because the NULL fields were skipped; with NULLs sorting last the
maximum is the `{NULL, NULL}` row.
## Are these changes tested?
Yes. `test_struct_and_map_partial_cmp` covers a struct with a list field,
NULL fields, mismatched field names, and maps that differ only in their values.
`aggregate.slt` adds `min` / `max` over structs with a list field and with a
NULL field. The `scalar` and `min_max` unit tests pass (with the one updated
expectation above).
## Are there any user-facing changes?
`min` / `max` and other scalar comparisons over structs with nested or NULL
fields, and over maps, return the correct ordering instead of an internal error
or a wrong result. A struct whose fields are NULL now sorts after one with
values in these comparisons.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
