CalvinKirs opened a new pull request, #70434: URL: https://github.com/apache/doris/pull/70434
### Background Nereids ReorderJoin extracts join/filter clusters into MultiJoin and then reconstructs the join tree. Top-down rewrite batches can retry a changed parent before visiting and normalizing its children. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: Join reconstruction can produce consecutive filters inside a child subtree. On the next parent retry, both the rule entry and recursive extraction assume that a filter has a direct join child, causing a ClassCastException. Require a direct join child at rule entry and preserve other filtered subtrees as join-cluster boundaries. Queries with these shapes can then be planned without disabling join reorder; most of the diff adds test coverage. ### Details - **Frequency:** Plan-dependent: a top-down parent retry can observe consecutive filters produced by an earlier reorder before child MergeFilters runs. - **Root cause:** The subtree pattern accepts filters as well as joins, while the entry action and recursive extractor cast a filter's child to LogicalJoin without establishing that shape. - **Why existing tests missed it:** They did not cover consecutive filters generated by an earlier reconstruction or the resulting parent retry. - **Fix:** Guard the rule entry with a direct-join-child predicate and retain non-direct-join filtered subtrees unchanged during recursive extraction. The original subtree pattern, volatile-expression barriers, outer-join predicate placement and hint keys are preserved. Production change: +4/-3 lines in one file. - **Other paths and branches:** The ordinary and MV rewrite batches share this rule. Unit tests cover batches with and without MergeFilters; source review traced both adaptive top-down scheduler implementations. This PR targets master. - **Behavior change:** Previously failing consecutive-filter plans can be optimized without a ClassCastException; no configuration or FE-BE protocol changes. ### Testing All checks below ran locally before formal source review: - Initial old-code negative control: 16 ReorderJoin tests, with four ClassCastException errors covering root, nested boundary, generated-chain retry and a top-down batch. The final harness was subsequently scoped to the relevant production batch to exclude an unrelated wrapper-added outer-to-anti rewrite, and a fifth test was added; the initial baseline is not an identical-final-harness comparison. - `./run-fe-ut.sh --run ReorderJoinTest,MergeFiltersTest,PushDownFilterThroughJoinTest,PushFilterInsideJoinTest,FindHashConditionForJoinTest`: 27 tests passed, including all 17 final ReorderJoin tests. New checks cover five join types, deterministic/volatile boundaries, generated-chain stability, both batch compositions and output-slot preservation. - `./build.sh --fe`: FE packaging and Checkstyle passed. The local FE-only build temporarily skipped unused native dependency bootstrap; build script/config changes were restored and are not part of this PR. - `./run-regression-test.sh --run -d nereids_rules_p0/reorder_join -s consecutive_filters,more_than_one_cross_join -parallel 1 -suiteParallel 1`: both suites passed with the current FE and an existing SelectDB 4.1.4 BE package; BE was not rebuilt. Compatibility settings were `be_exec_version=8` and `enable_local_shuffle_planner=false`, with logical join reorder enabled. - New regression results were generated with the standard runner and then verified by its normal comparison mode. All four populated queries independently matched SQLite 3.53.1 results, including NULL extension and duplicate-row multiplicities. ### Notes for Reviewers The SQL suite is semantic coverage; it is not claimed to reproduce the original customer SQL's failure with its full metadata. A fresh master BE and a cloud runtime were not tested. The final shallow unit plans use the default adaptive visitor route; the stack implementation was source-reviewed rather than separately forced in the final run. Formal local `doris-repo-review` completed against commit `02c22afbe248c8a6685228888eb58cb1ae62e56e`: two independent reviewers, all four changed files covered, no accepted findings, converged in round 1. ### Release note Fix query planning failures when nested join rewrites produce consecutive filters. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: unit tests, FE build/Checkstyle and local SQL regressions passed; fresh master BE, cloud runtime and exact customer metadata replay remain unverified. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: Production fix, unit and SQL regression tests, expected output generated by the regression runner, and this PR description. - Human review: Pending maintainer review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
