mrhhsg opened a new pull request, #68110:
URL: https://github.com/apache/doris/pull/68110
### What problem does this PR solve?
Issue Number: None
Problem Summary:
With `enable_prune_nested_column = true`, the planner splits a STRUCT
column's
access paths into predicate paths (read before filtering) and lazy paths
(read
only for surviving rows). SegmentIterator relies on this split when a common
expression references the column: it reads the column in the PREDICATE phase,
evaluates the expression, and recovers the lazy nested fields afterwards.
The TopN filter is a common expression that BE attaches to the scan at
runtime,
after the planner computed the access paths. When ORDER BY and WHERE
reference
different fields of the same STRUCT, for example
```sql
SELECT struct_element(s, 'a') FROM t
WHERE struct_element(s, 'b') IS NOT NULL
ORDER BY 1 NULLS LAST LIMIT 1;
```
the predicate paths only contain `s.b.NULL`, so `s.a` is a placeholder (NULL)
while the TopN filter `struct_element(s, 'a') <= current_top` is evaluated.
Every row of the later tablets is rejected and the global minimum is lost,
silently returning a wrong LIMIT result.
This PR marks columns referenced by runtime-generated common expressions
(TopN filters and runtime filters) and keeps them in the NORMAL read phase,
so
all access paths are materialized before the expression is evaluated. The
lazy
split is still used for planner-visible expressions.
### Release note
None
### Check List (For Author)
- Test:
- Unit Test: `SegmentIteratorRuntimeCommonExprTest` covers the lazy split
decision for planner expressions, TopN filters, and both on one column.
- Regression test: `topn_filter_nested_column_pruning` reproduces the
wrong
result with TIMESTAMPTZ/INT/DATE STRUCT fields, ASC/DESC, NULLS
FIRST/LAST,
selective predicates and multi-tablet scans.
- Behavior changed: No
- Does this need documentation: No
https://claude.ai/code/session_01E3gDAafAXZELs6AHVfD9UG
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
