FrankChen021 commented on code in PR #19633:
URL: https://github.com/apache/druid/pull/19633#discussion_r3505746033
##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/ReindexingDeletionRuleOptimizer.java:
##########
@@ -95,6 +95,31 @@ public DataSourceCompactionConfig optimizeConfig(
.build();
}
+ @Override
+ public boolean hasUnappliedDeletionRules(
+ DataSourceCompactionConfig config,
+ CompactionCandidate candidate,
+ CompactionJobParams params
+ )
+ {
+ if (config.getTransformSpec() == null) {
+ return false;
+ }
+ final DimFilter filter = config.getTransformSpec().getFilter();
+ if (!(filter instanceof NotDimFilter)) {
+ return false;
+ }
+
+ // Compare the decomposed NOT(OR(...)) clauses (as optimizeConfig does)
rather than the whole
+ // transformSpec, so a spec differing only by folded-in partitioning
virtual columns isn't mistaken
+ // for a pending deletion. Unlike optimizeConfig, never-compacted
candidates are not short-circuited.
+ return computeRequiredSetOfFilterRulesForCandidate(
Review Comment:
[P1] Treat mixed null fingerprints as unapplied deletion rules
This new mandatory-compaction check delegates to
computeRequiredSetOfFilterRulesForCandidate, which ignores null
indexingStateFingerprint values once any non-null fingerprint exists. If an
interval contains segments already fingerprinted with all deletion rules plus
newer or never-compacted null-fingerprint segments, this returns false, so
forcePendingDeletionCompaction does not bypass the policy thresholds. Below
minUncompactedCount/min bytes, those null-fingerprint segments are left
uncompacted and their deletion rules remain unapplied. Null fingerprints should
make this check true unless every segment proves the deletion rules were
applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
