Copilot commented on code in PR #68361:
URL: https://github.com/apache/doris/pull/68361#discussion_r4068242001
##########
be/src/cloud/cloud_cumulative_compaction.cpp:
##########
@@ -40,6 +47,74 @@ namespace doris {
#include "common/compile_check_begin.h"
using namespace ErrorCode;
+namespace cloud {
+
+bool is_single_rowset_compaction_candidate(const RowsetSharedPtr& rowset) {
+ const auto& rowset_meta = rowset->rowset_meta();
+ const int64_t overlap_unit_count =
+ rowset_meta->segments_overlap() == NONOVERLAPPING_WITHIN_GROUP
+ ?
static_cast<int64_t>(rowset_meta->segment_group_sizes().size())
+ : rowset->num_segments();
+ return !rowset_meta->has_delete_predicate() &&
rowset_meta->is_segments_overlapping() &&
+ overlap_unit_count >=
config::cloud_single_rowset_compaction_min_segments;
Review Comment:
For an already grouped rowset this uses the number of logical groups as the
eligibility threshold. After the first pass, a rowset can still have hundreds
or thousands of physical segments but only a small number of groups (for
example, 1,000 segments become about 16 groups with the default group size), so
this returns false once the group count is below
`cloud_single_rowset_compaction_min_segments`. The policy can still select that
singleton by its physical compaction score, and execution then falls back to an
ordinary merge that initializes all segments, recreating the memory spike this
feature is meant to avoid. Keep `NONOVERLAPPING_WITHIN_GROUP` rowsets on the
grouped path until the groups have been coalesced into `NONOVERLAPPING` (while
retaining the threshold for raw overlapping rowsets).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
