englefly opened a new pull request, #70100:
URL: https://github.com/apache/doris/pull/70100
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
A TopN runtime filter pushed into the left input of a RIGHT or FULL OUTER
JOIN can remove a matching row and cause the join to emit a new null-extended
row. Expressions such as `COALESCE(l.k, -1)` turn the generated NULL into a
non-NULL value that can sort ahead of the current bound, even with `NULLS
LAST`. With input keys 0 and 1, `ORDER BY COALESCE(l.k, -1), COALESCE(r.k, -1)
LIMIT 1` can consequently return `(-1, 1)` instead of `(0, 0)`.
Use the existing `NullToNonNullFunction.canConvertNullToNonNull()` contract
to check the complete probe expression before pushing through a join that can
null-extend its left side. Share the check between hash and nested loop joins,
and recheck rewritten hash join keys. Preserve the existing NULLS FIRST
restriction and safe pushdown on preserved join sides.
### Release note
Fix incorrect TopN query results for outer joins ordered by expressions that
convert NULL inputs into non-NULL values.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test
Validation:
- FE build: `bash build.sh --fe --exclude-obs-dependencies`.
- FE unit tests: `MAVEN_ARGS='-Ddisable.obs=true' bash run-fe-ut.sh --run
org.apache.doris.nereids.postprocess.TopNRuntimeFilterTest` — 13 tests passed.
The new visitor tests cover 52 expression, join, and NULL-order combinations,
including nested expressions and input-dependent `AlwaysNotNullable`
expressions.
- Regression: `bash run-regression-test.sh --run -d nereids_syntax_p0 -s
topn_filter_outer_join_null` — 82 result checks passed, including 40
enabled/disabled filter pairs and two bare-column NULL-order controls. Expected
output was generated with the regression runner. The new plan assertion fails
on the unpatched master build.
- FE Checkstyle: `mvn -Ddisable.obs=true checkstyle:check -pl fe-core` —
zero violations.
Optional Huawei OBS dependencies were excluded because they are unavailable
from the configured repositories. The reproduction and regression use native
OLAP tables.
- Behavior changed:
- [ ] No.
- [x] Yes. Unsafe TopN runtime filters are no longer pushed through
null-generating outer join sides.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
