github-actions[bot] commented on code in PR #65482:
URL: https://github.com/apache/doris/pull/65482#discussion_r3599901507
##########
be/src/exec/operator/olap_scan_operator.cpp:
##########
@@ -698,14 +698,23 @@ Status
OlapScanLocalState::_init_scanners(std::list<ScannerSPtr>* scanners) {
bool read_row_binlog =
p._olap_scan_node.__isset.read_row_binlog &&
p._olap_scan_node.read_row_binlog;
bool has_tso_predicate = _scan_ranges[0]->__isset.start_tso ||
_scan_ranges[0]->__isset.end_tso;
+ // Query cache incremental merge: the read sources captured in prepare()
+ // only cover the delta versions (cached_version, current_version], which
+ // is all the scanners need; no version plumbing is required here.
+ const bool cache_incremental =
+ _query_cache_decision != nullptr &&
+ _query_cache_decision->mode ==
QueryCacheInstanceDecision::Mode::INCREMENTAL;
// The flag of preagg's meaning is whether return pre agg data(or partial
agg data)
// PreAgg ON: The storage layer returns partially aggregated data without
additional processing. (Fast data reading)
// for example, if a table is select userid,count(*) from base table.
// And the user send a query like select userid,count(*) from base table
group by userid.
// then the storage layer do not need do aggregation, it could just return
the partial agg data, because the compute layer will do aggregation.
// PreAgg OFF: The storage layer must complete pre-aggregation and return
fully aggregated data. (Slow data reading)
- if (enable_parallel_scan && !p._should_run_serial &&
+ // Incremental merge deltas are small by construction, so the parallel
+ // scanner builder (which redistributes rowsets by size) is pointless for
+ // them; use plain scanners instead.
+ if (enable_parallel_scan && !cache_incremental && !p._should_run_serial &&
Review Comment:
[P2] Keep large incremental deltas parallel
`cache_incremental` bypasses `ParallelScannerBuilder` unconditionally, but
the delta is not actually small by construction.
`query_cache_max_incremental_merge_count` counts prior incremental write-backs,
not versions accumulated while an entry is idle, so a V50 entry with
`delta_count == 0` can remain unused until V550 and still capture a contiguous
`(50, 550]` suffix (including a large cumulative-compaction rowset wholly
inside that window). For a full-range aggregation the fallback loop below has
one condition range and creates only one scanner per tablet, while
`ParallelScannerBuilder` can split the already captured `_read_sources` by
rowset/segment rows. This can turn the largest stale-cache scans into serial
intra-tablet work. Please let the builder handle incremental read sources too,
or gate this bypass/fallback on an actual delta row/segment/byte bound, and
cover a dormant entry followed by a large suffix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
