matanper commented on issue #70381: URL: https://github.com/apache/doris/issues/70381#issuecomment-6725636689
A related gap on master, found while reviewing the patch (traced in code; not yet reproduced): Nested-column pruning does not see the expressions inside an OLAP scan's virtual columns. `PushDownVirtualColumnsIntoOlapScan` runs in `Rewriter` (`:810`) before `NestedColumnPruning` (`:951`). Access paths are collected only from plan expressions above the scan: `AccessPathPlanCollector.visitLogicalOlapScan` (`:374-385`) looks up `allSlotToAccessPaths` for the scan's output slots and never visits `getVirtualColumns()`. As a result, a STRUCT / ARRAY / VARIANT column that is read inside a virtual column gets no access path from that read. If the same column is also read elsewhere in the query through a different sub-path, pruning narrows it to that other sub-path. The virtual column expression then reads a subfield the scan no longer provides. Example shape, with `experimental_enable_virtual_slot_for_cse = true` and `s STRUCT<a INT, b INT>`: ```sql SELECT struct_element(s, 'a') FROM t WHERE abs(struct_element(s, 'b')) > 1 OR abs(struct_element(s, 'b')) < -5; ``` `abs(struct_element(s, 'b'))` becomes a virtual column, so the only path collected for `s` is `s.a`. Suggested fix: in `visitLogicalOlapScan`, also run the expression collector over each virtual column expression, so their paths are merged into the scan slots' access paths. This matters more once lambda / ARRAY expressions become candidates (see Solution above), because a repeated `ARRAY_MAP(..., REGEXP_EXTRACT_ALL(v['x'], ...))` next to a `SELECT v['y']` is a common shape. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
