LiaCastaneda opened a new pull request, #27086:
URL: https://github.com/apache/datafusion/pull/27086
## Which issue does this PR close?
- Part of #25017 (the Substrait round trip; dotted field access on lambda
parameters in the SQL planner is separate).
## Rationale for this change
A field access on a lambda parameter over a list of structs fails once the
plan goes through Substrait:
```sql
CREATE TABLE t AS SELECT [named_struct('a', 1, 'b', 'x')] AS s;
SELECT array_transform(s, x -> get_field(x, 'b')) FROM t;
```
The query runs fine on its own, but converting its plan to Substrait and
consuming it back fails:
```
Error during planning: Field b not found in struct
```
This happens because Substrait types carry no struct field names; only
relation schemas do. The plan declares the lambda parameter with a bare struct
type, so the consumer names its fields `c0, c1`, and `get_field(x, 'b')` looks
for a field that doesn't exist. The list `s` itself keeps its real names, as
they come from the table schema.
## What changes are included in this PR?
When the consumer builds a higher-order function, it now resolves the lambda
parameters from the function's arguments (`Expr::resolve_lambda_variables`), as
the `TypeCoercion` analyzer already does later. The parameter gets the list's
element type, field names included, before anything checks the body.
A function that reads a parameter of an enclosing lambda can't be resolved
on its own, because that parameter's type is only known to the enclosing
function. For example, the inner call in `array_transform(s, x ->
array_transform([1, 2], i -> get_field(x, 'a') + i))` reads `x`, so it is
resolved together with the outer call.
## What is the testing strategy for this PR?
`roundtrip_lambda_parameter_struct_field_names` in
`roundtrip_logical_plan.rs` round trips a struct field access on:
- a parameter over a column
- a parameter over a computed list
- the inner parameter of nested lambdas
- an outer parameter read inside an inner lambda
Without the change, the test fails with the error above. The existing lambda
tests, including the dictionary and string view ones, still pass.
## Are there any user-facing changes?
Substrait plans that access struct fields of lambda parameters now load. No
API changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
