gianm commented on code in PR #19091:
URL: https://github.com/apache/druid/pull/19091#discussion_r2907554821
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisorIOConfig.java:
##########
@@ -90,8 +91,11 @@ public SeekableStreamSupervisorIOConfig(
this.autoScalerEnabled = autoScalerConfig != null &&
autoScalerConfig.getEnableTaskAutoScaler();
// if autoscaler is enabled, then taskCount will be ignored here and
initial taskCount will equal to taskCountStart/taskCountMin
Review Comment:
This comment is no longer accurate, please fix it.
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/CostBasedAutoScaler.java:
##########
@@ -501,6 +510,56 @@ CostMetrics collectMetrics()
);
}
+ private static String constructCostTable(int[] taskCounts, CostResult[]
results)
+ {
+ final StringBuilder table = new StringBuilder();
+ table.append("Task count ");
+ for (int taskCount : taskCounts) {
+ table.append(StringUtils.format("| %8d", taskCount));
+ }
+ table.append("\nLag cost ");
+ for (CostResult result : results) {
+ table.append(StringUtils.format("| %8.1f", result.lagCost()));
+ }
+ table.append("\nIdle cost ");
+ for (CostResult result : results) {
+ table.append(StringUtils.format("| %8.1f", result.idleCost()));
+ }
+ table.append("\nTotal cost ");
+ for (CostResult result : results) {
+ table.append(StringUtils.format("| %8.1f", result.totalCost()));
+ }
+
+ return table.toString();
+ }
+
+ /**
+ * Checks if the given metrics are valid for auto-scaling. If they are not
+ * valid, auto-scaling will be skipped until fresh metrics are available.
+ *
+ * @return Either an error or a success boolean.
+ */
+ private Either<String, Boolean> validateMetricsForScaling(CostMetrics
metrics)
+ {
+ if (metrics == null) {
+ return Either.error("No metrics collected");
+ } else if (metrics.getAvgProcessingRate() < 0 ||
metrics.getPollIdleRatio() < 0) {
+ return Either.error("Task metrics not available");
+ } else if (metrics.getCurrentTaskCount() <= 0 ||
metrics.getPartitionCount() <= 0) {
+ return Either.error("Supervisor metrics not available");
+ } else if (metrics.getAvgPartitionLag() < 0) {
+ return Either.error("Lag metrics not available");
+ } else if (metrics.getAvgProcessingRate() < 1 &&
metrics.getAvgPartitionLag() > MAX_IDLENESS_PARTITION_LAG) {
+ log.makeAlert(
Review Comment:
After calling `makeAlert` you get an `AlertBuilder` that needs to have
`emit()` called on it. As written, it will not actually emit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
