starocean999 opened a new pull request, #67152:
URL: https://github.com/apache/doris/pull/67152
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
When a correlated subquery uses QUALIFY and references an outer column
inside the
QUALIFY clause, the Nereids analyzer mishandles that outer column in all four
`FillUpQualifyMissingSlot` plan shapes:
1. `Qualify(Aggregate)` / `Qualify(Having, Aggregate)` (explicit GROUP BY):
Under the default ONLY_FULL_GROUP_BY SQL mode, the outer column is
treated as an
inner non-grouped column and the query is rejected.
Reproduction:
```sql
SELECT o.k
FROM (
SELECT CAST(10 AS INT) AS k, CAST(1 AS INT) AS flag
UNION ALL
SELECT CAST(20 AS INT) AS k, CAST(0 AS INT) AS flag
) AS o
WHERE EXISTS (
SELECT i.k
FROM (
SELECT CAST(1 AS INT) AS k
UNION ALL
SELECT CAST(2 AS INT) AS k
) AS i
GROUP BY i.k
QUALIFY row_number() OVER (ORDER BY i.k) = 1
AND o.flag = 1
);
```
Before the fix this fails with:
`error 1105: QUALIFY expression 'flag' must appear in the GROUP BY clause
or be used in an aggregate function.`
After the fix it returns a single row `10`.
2. `Qualify(Project)` / `Qualify(Having, Project)` (no GROUP BY):
The correlated outer column is incorrectly pushed into the inner project's
output, which the inner query cannot produce. This later crashes in
`PushProjectIntoUnion` with a `NullPointerException` when the project is
pushed
into a UNION. The same query shape (without `GROUP BY`) reproduces this
NPE;
after the fix it also returns `10`.
Root cause:
- `BindExpression` binds the outer column from the enclosing outer Scope and
records
it in the outer Scope's correlated slots.
- In `FillUpQualifyMissingSlot`:
- The `FILL_UP_QUALIFY_AGGREGATE` and `FILL_UP_QUALIFY_HAVING_AGGREGATE`
rules built
the `Resolver` with `new Resolver(agg)` WITHOUT the outer scope (unlike
the
HAVING/SORT missing-slot paths which pass
`ctx.cascadesContext.getOuterScope()`).
Without it, the `Resolver` cannot tell a correlated outer slot from an
inner
missing group-by column, so under ONLY_FULL_GROUP_BY it throws.
- The `FILL_UP_QUALIFY_PROJECT` and `FILL_UP_QUALIFY_HAVING_PROJECT` rules
collect
missing slots in `createPlan` with `filter(s ->
!projectOutputSet.contains(s))`
which also does not exclude correlated outer slots (unlike
`FillUpMissingSlots.collectNotExistsSlotAndAggFunc`), so the outer
column gets
added to the inner project output.
The fix passes the outer scope into all four rules (via `thenApply(ctx ->
...)`):
- `FILL_UP_QUALIFY_AGGREGATE` / `FILL_UP_QUALIFY_HAVING_AGGREGATE`: the
`Resolver`
now receives `ctx.cascadesContext.getOuterScope()` and skips slots found
in the
outer scope's correlated slots, matching the normal missing-slot paths.
- `FILL_UP_QUALIFY_PROJECT` / `FILL_UP_QUALIFY_HAVING_PROJECT`: `createPlan`
now
receives the outer scope and filters correlated slots out of the project's
`notExistedInProject` (and the distinct-branch `missingSlots`), so outer
columns
are never pushed into the inner project's output.
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
