FrankChen021 commented on code in PR #19412:
URL: https://github.com/apache/druid/pull/19412#discussion_r3195590164
##########
server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java:
##########
@@ -448,6 +462,52 @@ public Map<String, AutoCompactionSnapshot>
getAutoCompactionSnapshot()
return autoCompactionSnapshotPerDataSource.get();
}
+ /**
+ * Reduces {@code maxNumTasks} on the task context for MSQ minor compactions
+ * by the percent specified under
+ * {@link ClientMSQContext#CTX_MINOR_COMPACTION_TASK_PERCENT} in the same
+ * context, defaulting to {@link #DEFAULT_MINOR_COMPACTION_TASK_PERCENT} when
+ * absent. The scaled value is floored at {@link
ClientMSQContext#DEFAULT_MAX_NUM_TASKS}
+ * (the MSQ minimum of 1 controller + 1 worker). No-op when the engine is
+ * native, the mode is full, or the percent is 100.
+ */
+ private static void maybeScaleMaxNumTasksForMinorCompaction(
+ Map<String, Object> context,
+ CompactionMode compactionMode,
+ ClientCompactionRunnerInfo compactionRunner
+ )
+ {
+ if (compactionMode != CompactionMode.UNCOMPACTED_SEGMENTS_ONLY
+ || !CompactionEngine.MSQ.equals(compactionRunner.getType())) {
+ return;
+ }
+
+ final int percent = QueryContext.of(context).getInt(
Review Comment:
[P2] Validate minorCompactionTaskPercent when accepting the compaction config
The new context key is only parsed and range-checked while creating a minor
MSQ compaction task. Supervisor config validation still accepts values like 0,
200, or a non-numeric string, so the API can persist an invalid supervisor and
it will later fail job creation repeatedly once a minor compaction candidate
appears. Please add validation alongside the existing MSQ maxNumTasks
validation paths, including CascadingReindexingTemplate if applicable, so bad
configs are rejected before scheduling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
