FrankChen021 commented on code in PR #19646:
URL: https://github.com/apache/druid/pull/19646#discussion_r3505744145
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/WeightedCostFunction.java:
##########
@@ -53,27 +53,28 @@ public class WeightedCostFunction
/**
* Minimum rate of processing for any task in records per second. This is
used
* as a placeholder if avg rate is not available to ensure that cost
computations
- * do not return infinitely large lag recovery times.
+ * do not return infinitely large lag recovery times, at the expense of
underestimating the lag cost.
*/
- static final double MIN_PROCESSING_RATE = 1_000;
+ static final double MIN_PROCESSING_RATE = 5_000;
Review Comment:
[P2] Measured low throughput is treated as 5k r/s
This new 5,000 r/s floor is applied with Math.max(avgProcessingRate,
MIN_PROCESSING_RATE), so any real measured rate between the old 1,000 r/s floor
and 5,000 r/s is now replaced with 5,000 when calculating lagRecoveryTime.
Slow-but-running supervisors understate backlog recovery time by up to 5x,
lowering lag cost and potentially preventing needed scale-up. The fallback
should only mask missing/zero rates, or the floor should not override valid low
measured rates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
