mrhhsg commented on code in PR #67140:
URL: https://github.com/apache/doris/pull/67140#discussion_r3860756516
##########
be/src/exec/operator/distinct_streaming_aggregation_operator.cpp:
##########
@@ -176,8 +195,13 @@ Status
DistinctStreamingAggLocalState::_distinct_pre_agg_with_serialized_key(
const uint32_t rows = (uint32_t)in_block->rows();
_distinct_row.clear();
- if (_parent->cast<DistinctStreamingAggOperatorX>()._is_streaming_preagg &&
low_memory_mode()) {
- _stop_emplace_flag = true;
+ auto& parent = _parent->cast<DistinctStreamingAggOperatorX>();
+ if (parent._is_streaming_preagg) {
+ const auto memory_limit = parent._memory_limit(state());
+ COUNTER_SET(_memory_use_limit, static_cast<int64_t>(memory_limit));
+ if (low_memory_mode() || (memory_limit > 0 && _memory_usage() >
memory_limit)) {
+ _stop_emplace_flag = true;
Review Comment:
Revisited: we decided to keep the permanent latch after all and reverted the
per-block decision — the extra state made
`_distinct_pre_agg_with_serialized_key()` noticeably harder to follow for a
benefit that is narrow in practice. Rationale: this operator already latches
forever on the two pre-existing switches (low reduction rate,
`low_memory_mode()`), and the "lowered then restored" workload-group scenario
overlaps with `low_memory_mode(true)` being raised for the whole group
(`workload_group_manager.cpp`, low-watermark path), which would latch it
anyway. The memory-triggered latch is therefore just a third trigger of
existing behaviour, and it is skipped entirely while a LIMIT is pushed down
(see the sibling thread). `StreamingAggOperatorX` keeps re-evaluating per block
as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
