matanper opened a new issue, #69930:
URL: https://github.com/apache/doris/issues/69930
### Search before asking
- [x] I had searched in the issues and found no similar issues.
### Version
4.1.x (the same code is on master)
### What's Wrong?
A query whose lambda reads both an outer column and a struct field of its
own argument fails to plan when the array comes from the other side of a join
(here an uncorrelated scalar subquery):
```sql
SELECT k,
STRUCT_ELEMENT(ARRAY_FIRST(p -> STRUCT_ELEMENT(p, 1) = k,
(SELECT ARRAY_ZIP(COLLECT_LIST(v), COLLECT_LIST(r))
FROM (SELECT 'a' AS v, 'x' AS r UNION ALL SELECT 'b', 'y') s)), 2)
AS h,
COUNT() AS c
FROM (SELECT 'a' AS k UNION ALL SELECT 'b') t
GROUP BY k;
-- errCode = 2, detailMessage = Cannot invoke
"org.apache.doris.analysis.Expr.getChildren()" because "root" is null
```
Without `GROUP BY` the error is `Cannot read field "type" because the return
value of "java.util.ArrayList.get(int)" is null`. `COLLECT_LIST(STRUCT(v, r))`
instead of `ARRAY_ZIP` fails the same way. It works when the lambda reads no
outer column, when the array is `ARRAY<STRING>` (`p -> p = k`), or with `SET
enable_prune_nested_column = false`.
Cause: `PushDownProject` (`PUSH_DOWN_PROJECT_THROUGH_JOIN`) moves a
`PreferPushDownProject` expression to the first child whose output contains all
of the expression's input slots:
```java
if (child.getOutputSet().containsAll(e.getInputSlots())) {
```
`Expression.getInputSlots()` leaves out lambda arguments (`ArrayItemSlot`),
so `element_at(p, 'col1')` inside the lambda has no input slots, and the check
passes for any child. The rewritten plan pushes it into the join's left side,
where `p` does not exist:
```
LogicalJoin ( type=CROSS_JOIN )
|--LogicalUnion ( outputs=[k#10, element_at(p, 'col1')#18],
| constantExprsList=[['a' AS `k`#1, element_at(p#16, 'col1')
AS `element_at(p, 'col1')`#22], ...] )
+--LogicalProject ( projects=[array_zip(...) AS `ARRAY_ZIP(...)`#13] )
```
`PhysicalPlanTranslator.visitPhysicalProject` then gets a null `Expr` for
`p#16`, and `Expr.extractSlots` throws.
### What You Expected?
The query returns `a, x, 1` and `b, y, 1`.
### How to Reproduce?
Run the query above on a default session (`enable_prune_nested_column =
true`). It needs no table.
### Anything Else?
A possible fix: in `PushDownProject`, don't push an expression that uses a
lambda argument it does not bind itself. A whole lambda function such as
`array_map(p -> element_at(p, 'a'), arr)` binds its own argument and can still
be pushed toward the side that produces `arr`.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
