mrhhsg commented on code in PR #66145:
URL: https://github.com/apache/doris/pull/66145#discussion_r3845473798
##########
be/src/storage/segment/segment.cpp:
##########
@@ -379,6 +393,25 @@ bool Segment::is_tso_placeholder_col(int cid, const
Schema& schema,
return cid == schema.tso_col_idx();
}
+std::optional<Field> Segment::get_read_time_constant_value(
+ int cid, const Schema& schema, const StorageReadOptions& read_options)
const {
+ if (read_options.version.first != read_options.version.second) {
+ return std::nullopt;
+ }
+ if (cid == schema.version_col_idx()) {
Review Comment:
Fixed in c6cdab84e54. `Segment::new_iterator()` now evaluates
`col_id_to_predicates` on `__DORIS_VERSION_COL__` against the read-time version
through a synthetic `[version, version]` zonemap (the same shape as the tso
placeholder branch), and `_get_row_ranges_from_conditions()` skips physical
page zonemaps for every read-time constant column via
`get_read_time_constant_value()`. Coverage: UT
`ReadyAtOpenVersionPredicateUsesReadTimeVersion` /
`ReadyAtOpenVersionPredicatePrunesMismatchedVersion`, plus regression case
`late_rf_storage_version_ready_at_open` where the filter waits at scan open and
is normalized into an ordinary column predicate.
##########
be/src/storage/segment/segment_iterator.cpp:
##########
@@ -1176,6 +1264,21 @@ Status
SegmentIterator::_get_row_ranges_from_conditions(RowRanges* condition_row
_opts.stats->rows_stats_filtered +=
(pre_expr_zonemap_size - condition_row_ranges->count());
}
+ if (!_late_runtime_filter_ctxs.empty() &&
!condition_row_ranges->is_empty()) {
+ const auto rows_before_zonemap = condition_row_ranges->count();
+
RETURN_IF_ERROR(_apply_expr_zonemap_to_row_ranges(_late_runtime_filter_ctxs, 0,
+
condition_row_ranges));
+ const auto rows_after_zonemap = condition_row_ranges->count();
+ DORIS_CHECK_LE(rows_after_zonemap, rows_before_zonemap);
+ // This is a coarse informational count based on the condition
ranges. It may include
+ // rows already removed from _row_bitmap by other indexes, or rows
later excluded by
+ // delete bitmaps and parallel-scan ranges, so it can overestimate
the rows this
+ // iterator would actually read. The direct delta avoids extra
range and bitmap
+ // operations in the scan hot path while still reflecting late RF
ZoneMap pruning.
+ const auto rows_filtered = rows_before_zonemap -
rows_after_zonemap;
+ _opts.stats->rows_stats_filtered += rows_filtered;
+ _opts.stats->rows_late_runtime_filter_zonemap_filtered +=
rows_filtered;
Review Comment:
Fixed in c6cdab84e54. The initial late-RF ZoneMap counter now measures
removed live candidates: the condition ranges are intersected with
`_row_bitmap`, the parallel-scan `row_ranges` and the segment delete bitmap
before and after pruning, and only that delta goes to
`RowsLateRuntimeFilterZoneMapFiltered`. `rows_stats_filtered` keeps the coarse
range delta shared by the other zonemap paths. The two tests that encoded the
old behaviour were renamed to
`InitialLateRuntimeFilterPagePruningIgnoresUnassignedRows` /
`IgnoresDeletedRows` and now expect 0.
##########
be/src/storage/segment/segment_iterator.cpp:
##########
@@ -3297,22 +3438,127 @@ Status
SegmentIterator::current_block_row_locations(std::vector<RowLocation>* bl
return Status::OK();
}
+bool SegmentIterator::_has_arrived_late_runtime_filter() const {
+ return _late_runtime_filter_container != nullptr &&
+
_late_runtime_filter_container->arrived_cnt.load(std::memory_order_relaxed) > 0;
+}
+
+Status SegmentIterator::_install_late_runtime_filter(const VExprContextSPtrs&
expr_group,
+ VExprContextSPtrs*
installed_contexts) {
+ DORIS_CHECK(installed_contexts != nullptr);
+ DORIS_CHECK(!expr_group.empty());
+
+ VExprContextSPtrs local_contexts;
+ local_contexts.reserve(expr_group.size());
+ for (const auto& base_context : expr_group) {
+ DORIS_CHECK(base_context != nullptr);
+ VExprContextSPtr local_context;
+ RETURN_IF_ERROR(base_context->clone(_opts.runtime_state,
local_context));
Review Comment:
Not changed on purpose. `VExprContext::clone()` sharing `_root` and
re-running `open(THREAD_LOCAL)` is the existing mechanism for every
scanner-side clone, including the late-RF residual path on master
(`Scanner::try_append_late_arrival_runtime_filter` -> `clone_conjunct_ctxs()`),
which already runs while other scanner threads execute the same root. The only
root members written by a THREAD_LOCAL open are `_open_finished` and
`_is_args_all_constant`, both idempotent same-value stores; per-context state
(FunctionContexts, RF selectivity) is owned by the clone. Giving each
SegmentIterator an independently prepared tree would need a RowDescriptor that
the storage layer does not have, and would diverge from the scanner path
without removing the pattern. If the VExpr framework later moves open-state
into the context, both paths benefit together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
