Ishaan400 opened a new pull request, #26199:
URL: https://github.com/apache/datafusion/pull/26199
## Which issue does this PR close?
- Closes #26197.
## Rationale for this change
`map[key]` returns an error when the key literal's type differs from the
map's key type. The most common case is a map declared in SQL as `MAP(VARCHAR,
...)`: `VARCHAR` maps to `Utf8View`, so `m['a']` fails with `The key type Utf8
does not match the map key type Utf8View`. `m[1]` on a `MAP(INT, ...)` column
fails the same way (`Int64` vs `Int32`). `map_extract(m, 'a')` and
`element_at(m, 'a')` already work on the same columns.
## What changes are included in this PR?
`GetFieldFunc::coerce_types` now coerces each map lookup key to the map's
key type, the same way `map_extract` does. It follows the argument list through
nested maps (`n['a']['aa']`) and stops at the first non-map level, because
struct field names are only known from the literal values. NULL keys, nested
key types, and keys that already match apart from dictionary encoding are left
as they are, so those paths behave as before.
Since the coerced key is a cast of a literal, the simplifier folds it into a
literal of the map key type (for example `get_field(t.m, Utf8View("a"))`).
Output column names are unchanged (`t.m[a]`).
`strip_dictionary` in `utils.rs` is now `pub(crate)` so `get_field` can
reuse it.
## What is the testing strategy for this PR?
New cases at the end of `map.slt`:
- `VARCHAR` keys: projection, missing key, NULL map row, filters, and the
same result as `map_extract` / `element_at`
- `INT` and `SMALLINT` keys with an `Int64` literal, including a missing key
- nested `MAP(VARCHAR, MAP(VARCHAR, INT))` lookups, in a projection and a
filter
- an `Int32`-keyed map with struct values (`st[1]['x']`): the key is
coerced, the field name is not
- a dictionary-encoded `Utf8` key map, which already worked and still does
All cases except the dictionary one fail on `main`.
## Are there any user-facing changes?
Yes: `map[key]` now works when the key literal's type differs from the map
key type. No public API changes.
Because the key is coerced the same way as in `map_extract`, a key that
cannot be cast to the map key type now fails at planning time with a cast
error, as `map_extract` does (for example `m['x']` on an `INT`-keyed map, or
`3000000000` on an `Int32`-keyed map). Before this change, those queries also
failed, with the key-type mismatch error. A lossy cast also behaves like
`map_extract`: `m[1.5]` on an `INT`-keyed map looks up key `1`. I kept this
consistent with `map_extract` rather than giving `map[key]` its own rules.
Happy to change it if reviewers prefer a stricter rule.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
