sunchao commented on code in PR #5452:
URL: https://github.com/apache/datafusion-comet/pull/5452#discussion_r3857524849
##########
native/core/src/execution/planner.rs:
##########
@@ -3384,6 +3384,16 @@ impl PhysicalPlanner {
.collect::<Result<Vec<_>, _>>()?;
let fun_name = &expr.func;
+ // `map_entries` reuses the input map's entries array but declares the
entry `value` field
+ // nullable, so widen the argument first or Arrow rejects the child
type. See
+ // `coerce_child_fields_nullable`.
+ let args = if fun_name == "map_entries" {
+ args.into_iter()
+ .map(|arg| Self::coerce_child_fields_nullable(arg,
&input_schema))
Review Comment:
[P2] Widen only the outer map_entries value field
This recursive cast also changes nullability inside the map's key/value
types, although `map_entries` only needs the outer entry's `value` field to be
nullable. With normal folding and Parquet `id INT` values `1, 2, 3`:
```sql
SELECT id, array(
map_entries(map(1, IF(id = 1, map(1, 2), NULL)))[0].value,
map(2, coalesce(id, 0))) AS a
FROM t
```
Both Catalyst array children are `MapType(INT, INT, false)`, so the array
guard accepts them. The outer map already has `valueContainsNull=true`, and the
pre-change native query returns all three rows correctly. This cast
unnecessarily widens the *inner* map to `valueContainsNull=true`; `ListExtract`
and `GetStructField` preserve that changed type while the sibling remains
`false`. DataFusion 54.1 cannot coerce those unequal map types, and
`make_array` panics with `Arrays with inconsistent types passed to
MutableArrayData`.
Please preserve the nested key/value types and widen only the outer
entry-value flag. This is distinct from the original `map_entries` child-type
failure: that call succeeds here, and the new cast introduces a mismatch in its
downstream consumer. Verified with a passing native SQL control and a
differential kernel probe using the exact cast helper, DataFusion's
cast/map_entries/make_array, and Comet's extraction kernels; this was not a
full current-head native rebuild.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
