kfaraz commented on code in PR #19687:
URL: https://github.com/apache/druid/pull/19687#discussion_r3755888338
##########
indexing-service/src/main/java/org/apache/druid/indexing/overlord/supervisor/SupervisorManager.java:
##########
@@ -644,6 +648,90 @@ public boolean isAnotherTaskGroupPublishingToPartitions(
}
}
+ /**
+ * Simulates the effects of the {@code costBased} auto-scaler by computing
the optimal
+ * task count under various values of aggregate lag.
+ */
+ public Map<String, Object> simulateAutoscaling(
+ String supervisorId,
+ CostBasedAutoScalerConfig config,
+ int maxProcessingRatePerTask,
+ @Nullable Integer requestedTaskCount
+ )
+ {
+ // Validate that this is a SeekableStreamSupervisor
+ final Pair<Supervisor, SupervisorSpec> supervisorPair =
supervisors.get(supervisorId);
+ if (supervisorPair == null || supervisorPair.rhs == null ||
supervisorPair.lhs == null) {
+ throw NotFound.exception("Invalid supervisor[%s]", supervisorId);
+ } else if (!(supervisorPair.rhs instanceof SeekableStreamSupervisorSpec)) {
+ throw InvalidInput.exception(
+ "Cannot simulate autoscaling for supervisor[%s] of type[%s]",
+ supervisorId, supervisorPair.rhs.getType()
+ );
+ }
+
+ final SeekableStreamSupervisorSpec supervisorSpec =
(SeekableStreamSupervisorSpec) supervisorPair.rhs;
+
+ // Validate the inputs
+ final long criticalLag =
Configs.valueOrDefault(config.getCriticalLagThreshold(), 1_000_000);
+ InvalidInput.conditionalException(
+ criticalLag >= 1000,
+ "Value of critical lag[%d] must be 1000 or more",
+ criticalLag
+ );
+ InvalidInput.conditionalException(
+ maxProcessingRatePerTask >= 100,
+ "Value of maxProcessingRatePerTask[%d] must be 100 events per second
or more",
+ maxProcessingRatePerTask
+ );
+ InvalidInput.conditionalException(
Review Comment:
Added validation in `CostBasedAutoScalerConfig` itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
