Copilot commented on code in PR #67930:
URL: https://github.com/apache/doris/pull/67930#discussion_r4002060384
##########
be/src/cloud/cloud_cumulative_compaction.cpp:
##########
@@ -40,6 +47,74 @@ namespace doris {
#include "common/compile_check_begin.h"
using namespace ErrorCode;
+namespace cloud {
+
+bool is_single_rowset_compaction_candidate(const RowsetSharedPtr& rowset) {
+ const auto& rowset_meta = rowset->rowset_meta();
+ const int64_t overlap_unit_count =
+ rowset_meta->segments_overlap() == NONOVERLAPPING_WITHIN_GROUP
+ ?
static_cast<int64_t>(rowset_meta->segment_group_sizes().size())
+ : rowset->num_segments();
Review Comment:
Counting only the number of logical groups makes an already-grouped rowset
with many physical segments but fewer than
`cloud_single_rowset_compaction_min_segments` groups ineligible.
`pick_from_candidates` then falls back to ordinary one-rowset compaction; when
vertical compaction is disabled, `VMergeIterator::init()` initializes an
iterator for every segment, reintroducing the memory spike this feature is
intended to prevent. Keep grouped rowsets on the bounded grouped path based on
their physical segment count (or otherwise prevent this fallback until the
rowset is fully non-overlapping).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
