FrankChen021 commented on code in PR #19975:
URL: https://github.com/apache/druid/pull/19975#discussion_r3843471588
##########
server/src/main/java/org/apache/druid/server/compaction/CompactionStatusTracker.java:
##########
@@ -97,13 +97,17 @@ public CompactionStatus computeCompactionStatus(
return status;
}
- // Skip intervals that have been filtered out by the policy
+ // Exclude intervals that have been filtered out by the policy
final Eligibility eligibility
= searchPolicy.checkEligibilityForCompaction(candidate,
lastTaskStatus);
if (eligibility.isEligible()) {
return CompactionStatus.pending("Not compacted yet");
} else {
- return CompactionStatus.skipped("Rejected by search policy: %s",
eligibility.getReason());
+ return CompactionStatus.skipped(
Review Comment:
[P2] Policy-rejected remainder remains pending
Although this new branch tags policy-filtered candidates as
REJECTED_BY_SEARCH_POLICY, CompactSegments only runs computeCompactionStatus
while task slots are available. When slots are exhausted, the remaining
iterator is drained directly into pending stats, so these same policy-rejected
intervals are reported as awaiting compaction and omitted from the new
skipped-reason breakdown. Apply status/policy evaluation while draining the
remainder, or otherwise classify it before addToPending.
##########
server/src/main/java/org/apache/druid/server/compaction/CompactionSnapshotBuilder.java:
##########
@@ -101,8 +125,19 @@ private void collectSnapshotStats(AutoCompactionSnapshot
autoCompactionSnapshot)
stats.add(Stats.Compaction.COMPACTED_BYTES, rowKey,
autoCompactionSnapshot.getBytesCompacted());
stats.add(Stats.Compaction.COMPACTED_SEGMENTS, rowKey,
autoCompactionSnapshot.getSegmentCountCompacted());
stats.add(Stats.Compaction.COMPACTED_INTERVALS, rowKey,
autoCompactionSnapshot.getIntervalCountCompacted());
- stats.add(Stats.Compaction.SKIPPED_BYTES, rowKey,
autoCompactionSnapshot.getBytesSkipped());
- stats.add(Stats.Compaction.SKIPPED_SEGMENTS, rowKey,
autoCompactionSnapshot.getSegmentCountSkipped());
- stats.add(Stats.Compaction.SKIPPED_INTERVALS, rowKey,
autoCompactionSnapshot.getIntervalCountSkipped());
+
+ // Skipped stats are emitted per reason. The total for a datasource is the
sum
+ // across all values of the 'reason' dimension. The 'category' dimension
allows
+ // alerting on a class of reasons without enumerating the reasons
themselves.
+ for (CompactionSkipStatistics skipStats :
autoCompactionSnapshot.getSkippedStatsByReason()) {
+ final RowKey skipRowKey = RowKey
Review Comment:
[P2] Default metrics drop new skip dimensions
These new stats are emitted with dataSource, reason, and category row keys,
but the default emitter configs still map segment/skipCompact/* and
interval/skipCompact/count to dataSource only. Prometheus therefore collapses
all reasons into one series, and StatsD filters reason and category, so the
advertised breakdown is silently unavailable under default configs. Update the
default mappings alongside the new dimensions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
