asdf2014 commented on code in PR #65482:
URL: https://github.com/apache/doris/pull/65482#discussion_r3600042991
##########
be/src/exec/operator/olap_scan_operator.cpp:
##########
@@ -698,14 +698,23 @@ Status
OlapScanLocalState::_init_scanners(std::list<ScannerSPtr>* scanners) {
bool read_row_binlog =
p._olap_scan_node.__isset.read_row_binlog &&
p._olap_scan_node.read_row_binlog;
bool has_tso_predicate = _scan_ranges[0]->__isset.start_tso ||
_scan_ranges[0]->__isset.end_tso;
+ // Query cache incremental merge: the read sources captured in prepare()
+ // only cover the delta versions (cached_version, current_version], which
+ // is all the scanners need; no version plumbing is required here.
+ const bool cache_incremental =
+ _query_cache_decision != nullptr &&
+ _query_cache_decision->mode ==
QueryCacheInstanceDecision::Mode::INCREMENTAL;
// The flag of preagg's meaning is whether return pre agg data(or partial
agg data)
// PreAgg ON: The storage layer returns partially aggregated data without
additional processing. (Fast data reading)
// for example, if a table is select userid,count(*) from base table.
// And the user send a query like select userid,count(*) from base table
group by userid.
// then the storage layer do not need do aggregation, it could just return
the partial agg data, because the compute layer will do aggregation.
// PreAgg OFF: The storage layer must complete pre-aggregation and return
fully aggregated data. (Slow data reading)
- if (enable_parallel_scan && !p._should_run_serial &&
+ // Incremental merge deltas are small by construction, so the parallel
+ // scanner builder (which redistributes rowsets by size) is pointless for
+ // them; use plain scanners instead.
+ if (enable_parallel_scan && !cache_incremental && !p._should_run_serial &&
Review Comment:
Fixed in 2033145942b. You are right that nothing bounded the delta: the
merge-count threshold limits prior write-backs, not idle accumulation, so the
"small by construction" comment was wrong and the bypass punished exactly the
queries with the largest suffixes. The bypass is removed rather than gated: the
builder already consumes the captured read sources as they are (each partial
read source carries the entire source's delete predicates and delete bitmap,
and the rowset readers are cloned per scanner), so the incremental delta now
goes through the same rowset/segment-row splitting as a full capture, and a
genuinely small delta still collapses to a single scanner through
min_rows_per_scanner, which is the actual size gate the old bypass was
approximating from outside. The regression suite gains the requested
dormant-entry phase: the entry is filled, thirty loads land on a single-tablet
partition while nothing queries it, and the first stale query is then asserted
to take the incr
emental path (stale-hit counter) and to agree with the uncached result, with
parallel_scan_min_rows_per_scanner lowered so the builder really splits that
delta into many scanners.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
