github-actions[bot] commented on code in PR #65242:
URL: https://github.com/apache/doris/pull/65242#discussion_r3527065846
##########
be/src/storage/segment/segment_iterator.cpp:
##########
@@ -1471,10 +1464,16 @@ bool SegmentIterator::_need_read_data(ColumnId cid) {
if (unique_id < 0) {
unique_id = column.parent_unique_id();
}
- if ((_need_read_data_indices.contains(cid) &&
!_need_read_data_indices[cid] &&
- !_output_columns.contains(unique_id)) ||
- (_need_read_data_indices.contains(cid) &&
!_need_read_data_indices[cid] &&
- _output_columns.count(unique_id) == 1 &&
+ // A column can skip data reads when its predicates have already been
fully resolved:
+ // either by an index, or by the segment zone map proving all predicates
on this column are
+ // always true and removing them before iterator initialization.
+ const bool used_by_common_expr =
+ cid < _is_common_expr_column.size() && _is_common_expr_column[cid];
+ const bool no_need_read_filter_column =
+ (_need_read_data_indices.contains(cid) &&
!_need_read_data_indices[cid]) ||
+ (_opts.zonemap_always_true_pred_cols.contains(cid) &&
!used_by_common_expr);
Review Comment:
This also affects key predicates. When a segment zonemap proves a key
predicate always true, this code records the key cid in
`zonemap_always_true_pred_cols`, but the original `key_ranges` are still
present. `_lazy_init()` applies `_get_row_ranges_by_keys()` before lazy
materialization setup, and the short-key binary search goes through
`_seek_and_peek()` -> `_seek_columns()` / `_read_columns()`. Those now call
`_need_read_data()` / `_prune_column()`, so a key cid in this set can be
skipped and filled with default placeholders;
`_compare_short_key_with_seek_block()` then compares the scan key against the
placeholder instead of the real segment key.
For example, a DUP-key segment with key values `[10, 20]` and a pushed range
`k >= 5 AND k < 30` can have the `k` predicates removed as zonemap-always-true,
while the short-key lookup still needs the real `k` values to compute row
bounds. Please keep key-range seek reads independent of this zonemap no-read
marker, or disable/remove the matching key range when the segment zonemap has
already proven it fully true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
