kfaraz commented on code in PR #20708:
URL: https://github.com/apache/druid/pull/20708#discussion_r4379367955
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/CostBasedAutoScaler.java:
##########
@@ -463,6 +463,29 @@ static int[] computeValidTaskCounts(
return result.toIntArray();
}
+ /**
+ * Extracts the average processing rate (records per second) from the task
stats.
+ *
+ * @param taskStats the stats map from supervisor.getStats()
+ */
+ static double extractAvgProcessingRate(Map<String, Map<String, Object>>
taskStats)
+ {
+ if (taskStats == null || taskStats.isEmpty()) {
+ return -1.;
+ }
+
+ // First try using the running average if available
+ final double runningAverage = extractAutoscalerMetric(
+ taskStats,
+ SeekableStreamIndexTaskRunner.PROCESSING_RATE_RUNNING_AVG
+ );
+ if (runningAverage >= 0) {
Review Comment:
I thought about this some more. It is cleaner to choose just one variant of
the metric. Otherwise, it becomes difficult to reason about the metric values
and the subsequent auto-scaler decisions.
If even one task reports the newer one, let's stick to that. Otherwise,
stick to the older metric.
The mixed version state would last only until the old tasks finish off.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
