Mryange opened a new pull request, #70460: URL: https://github.com/apache/doris/pull/70460
### Background Nereids pushes selected complex-type expressions into child projections to enable column pruning. Outer joins extend the nullable side with NULL values for unmatched rows. ### What problem does this PR solve? Issue Number: close #70448 Related PR: None PR Type: Bugfix Problem Summary: A projection such as `array_map(x -> upper(x), coalesce(b.arr, ['n']))` over the nullable side of a LEFT JOIN returned NULL for unmatched rows instead of `["N"]`. Require a context-independent NULL-input proof before moving such expressions below an outer join, and expose wrapped array inputs through a shared NULL-propagation interface. Most of this change adds tests for related array and map functions. ### Details - **Change map:** 106 changed production lines, 491 unit/regression test lines, and a 53-line generated regression baseline; no documentation changes. Production changes cover the outer-join proof and Lambda scope boundary, shared NULL-propagation metadata, and the three functions with wrapped array inputs. - **Frequency:** Queries using eligible complex-type expressions with NULL-replacing defaults over an outer join's nullable side. - **Root cause:** Precomputing the expression below the join replaced its result with NULL on unmatched rows, even when evaluating the expression on NULL inputs produces a default value. - **Why existing tests missed it:** Existing Lambda/pruning coverage did not combine outer-join null extension with NULL-replacing array/map inputs. - **Fix:** Substitute typed NULLs for input slots and reuse FE-only constant folding as proof. Expose semantic inputs for ArrayMap, comparator ArraySort and ArraySortedMerge, preserving invalid merge-flag errors. Keep Lambda-local expressions inside their scope when an outer expression cannot move. - **Other paths and branches:** Project and Filter share the proof; ON expressions retain their existing before-null-extension path. Preserved sides, inner joins and proven NULL-propagating expressions retain pushdown. Full-commit clean-pick checks against refreshed branch-4.1 and branch-4.2 both conflict because ArraySortedMerge is absent; 4.1 also reports a regression-data directory rename split. No automatic backport labels requested. - **Behavior change:** Unmatched outer-join rows retain the results of NULL-replacing expressions. ### Testing - ASAN BE and FE builds on the base master succeeded; the final modified FE build and Checkstyle passed. - Generated the new `outer_join_project_null_extension` baseline through the regression framework, then reran it in comparison mode: passed. The original unmatched row now returns `["N"]` and `["n"]` for mapped and filtered defaults. - Existing `lambda_null_pruning`, `test_map_lambda`, `test_array_sort_lambda_comparator` and `test_array_sorted_merge` regression suites passed. - `run-fe-ut.sh --run` for ArrayMapNullFoldingTest, PushDownProjectOuterJoinTest, PushDownProjectTest and FoldConstantTest: 44 tests passed. No full test run or performance benchmark claimed. ### Release note Fix incorrect NULL results and missing rows when array/map expressions with default values are pushed below outer joins. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex (GPT-6) - Generated: Implementation, tests and PR description. - Human review: The author reviewed the design and earlier code; final additions await human review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
