cecemei commented on code in PR #18968:
URL: https://github.com/apache/druid/pull/18968#discussion_r2765199112
##########
server/src/main/java/org/apache/druid/server/compaction/DataSourceCompactibleSegmentIterator.java:
##########
@@ -329,17 +331,17 @@ private void
findAndEnqueueSegmentsToCompact(CompactibleSegmentIterator compacti
continue;
}
- final CompactionCandidate candidates =
CompactionCandidate.from(segments, config.getSegmentGranularity());
- final CompactionStatus compactionStatus =
CompactionStatus.compute(candidates, config, fingerprintMapper);
- final CompactionCandidate candidatesWithStatus =
candidates.withCurrentStatus(compactionStatus);
+ final CompactionCandidate candidatesWithStatus =
Review Comment:
why should it be jobQueue only? both jobQueue and not as a coordinator duty?
##########
server/src/main/java/org/apache/druid/server/compaction/CompactionCandidateSearchPolicy.java:
##########
@@ -64,18 +62,27 @@ Eligibility checkEligibilityForCompaction(
*/
class Eligibility
{
- public static final Eligibility OK = new Eligibility(true, null);
+ public enum PolicyEligibility
Review Comment:
this is done
##########
server/src/main/java/org/apache/druid/server/compaction/CompactionCandidateSearchPolicy.java:
##########
@@ -64,18 +62,27 @@ Eligibility checkEligibilityForCompaction(
*/
class Eligibility
{
- public static final Eligibility OK = new Eligibility(true, null);
+ public enum PolicyEligibility
+ {
+ FULL_COMPACTION,
+ INCREMENTAL_COMPACTION,
+ NOT_ELIGIBLE,
+ NOT_APPLICABLE
Review Comment:
they are tracked differently in `AutoCompactionSnapshot`, my original
implementation only has three states and it failed:
https://github.com/apache/druid/actions/runs/21509820801/job/61973864065.
eventually i'd want evaluate() to only return eligibility and not status, so
complete/pending status should have different eligibility
##########
server/src/main/java/org/apache/druid/server/compaction/CompactionCandidateSearchPolicy.java:
##########
@@ -64,18 +62,27 @@ Eligibility checkEligibilityForCompaction(
*/
class Eligibility
Review Comment:
this is done
##########
server/src/main/java/org/apache/druid/server/compaction/CompactionCandidate.java:
##########
@@ -160,21 +169,122 @@ public CompactionStatus getCurrentStatus()
return currentStatus;
}
+ @Nullable
+ public CompactionCandidateSearchPolicy.Eligibility getPolicyEligibility()
+ {
+ return policyEligiblity;
+ }
+
/**
* Creates a copy of this CompactionCandidate object with the given status.
*/
public CompactionCandidate withCurrentStatus(CompactionStatus status)
{
- return new CompactionCandidate(segments, umbrellaInterval,
compactionInterval, numIntervals, status);
+ return new CompactionCandidate(
+ segments,
+ umbrellaInterval,
+ compactionInterval,
+ numIntervals,
+ policyEligiblity,
+ status
+ );
+ }
+
+ public CompactionCandidate
withPolicyEligibility(CompactionCandidateSearchPolicy.Eligibility eligibility)
+ {
+ return new CompactionCandidate(
+ segments,
+ umbrellaInterval,
+ compactionInterval,
+ numIntervals,
+ eligibility,
+ currentStatus
+ );
+ }
+
+ /**
+ * Evaluates this candidate for compaction eligibility based on the provided
+ * compaction configuration and search policy.
+ * <p>
+ * This method first evaluates the candidate against the compaction
configuration
+ * using a {@link CompactionStatus.Evaluator} to determine if any segments
need
+ * compaction. If segments are pending compaction, the search policy is
consulted
+ * to determine the type of compaction:
+ * <ul>
+ * <li><b>NOT_ELIGIBLE</b>: Returns a candidate with status SKIPPED,
indicating
+ * the policy decided compaction should not occur at this time</li>
+ * <li><b>FULL_COMPACTION</b>: Returns this candidate with status PENDING,
+ * indicating all segments should be compacted</li>
+ * <li><b>INCREMENTAL_COMPACTION</b>: Returns a new candidate containing only
+ * the uncompacted segments (as determined by the evaluator), with status
+ * PENDING for incremental compaction</li>
+ * </ul>
+ *
+ * @param config the compaction configuration for the datasource
+ * @param searchPolicy the policy used to determine compaction eligibility
+ * @return a CompactionCandidate with updated status and potentially
filtered segments
+ */
+ public CompactionCandidate evaluate(
Review Comment:
synced offline, DataSourceCompactibleSegmentIterator would be responsible
for filtering out ineligible candidates
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
