kfaraz commented on code in PR #19655:
URL: https://github.com/apache/druid/pull/19655#discussion_r3672291672
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/WeightedCostFunction.java:
##########
@@ -37,10 +37,30 @@ public class WeightedCostFunction
private static final Logger log = new Logger(WeightedCostFunction.class);
/**
- * Multiplier for a lag amplification factor; it was carefully chosen
- * during extensive testing as the most balanced multiplier for high-lag
recovery.
+ * Normal-path lag amplification multiplier. Critical-lag tiers provide the
+ * urgency amplification, so normal lag uses unamplified recovery time.
*/
- static final double LAG_AMPLIFICATION_MULTIPLIER = 0.3;
+ static final double LAG_AMPLIFICATION_MULTIPLIER = 0.0;
+
+ /**
+ * Amplification multiplier used once aggregate lag crosses {@link
#CRITICAL_LAG_TIER1_FRACTION} of
+ * {@link CostBasedAutoScalerConfig#getCriticalLagThreshold()} (tier 1 of
the critical-lag fast path).
+ */
+ static final double CRITICAL_LAG_AMPLIFICATION_MULTIPLIER = 6.0;
+
+ /**
+ * Fraction of {@link CostBasedAutoScalerConfig#getCriticalLagThreshold()}
at which tier 1 of the
+ * critical-lag fast path engages: amplification maxes out at {@link
#CRITICAL_LAG_AMPLIFICATION_MULTIPLIER}
+ * and the scale-up candidate boundary is bypassed.
+ */
+ static final double CRITICAL_LAG_TIER1_FRACTION = 0.75;
+
+ /**
+ * Fraction of {@link CostBasedAutoScalerConfig#getCriticalLagThreshold()}
at which tier 2 of the
+ * critical-lag fast path engages: the cost-minimization search is skipped
entirely and the task
+ * count jumps straight to the maximum.
+ */
+ static final double CRITICAL_LAG_TIER2_FRACTION = 0.95;
Review Comment:
Let's have tier 2 at 100% itself.
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/CostBasedAutoScalerConfig.java:
##########
@@ -46,8 +46,9 @@ public class CostBasedAutoScalerConfig implements
AutoScalerConfig
{
static final double DEFAULT_LAG_WEIGHT = 0.4;
static final double DEFAULT_IDLE_WEIGHT = 0.6;
- static final Duration DEFAULT_MIN_SCALE_UP_DELAY =
Duration.standardMinutes(10);
- static final Duration DEFAULT_MIN_SCALE_DOWN_DELAY =
Duration.standardMinutes(30);
+ static final Duration DEFAULT_MIN_SCALE_UP_DELAY =
Duration.standardMinutes(15);
+ static final Duration DEFAULT_MIN_SCALE_DOWN_DELAY =
Duration.standardMinutes(20);
Review Comment:
Let's keep scale down delay at 30 mins only.
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/WeightedCostFunction.java:
##########
@@ -37,10 +37,30 @@ public class WeightedCostFunction
private static final Logger log = new Logger(WeightedCostFunction.class);
/**
- * Multiplier for a lag amplification factor; it was carefully chosen
- * during extensive testing as the most balanced multiplier for high-lag
recovery.
+ * Normal-path lag amplification multiplier. Critical-lag tiers provide the
+ * urgency amplification, so normal lag uses unamplified recovery time.
*/
- static final double LAG_AMPLIFICATION_MULTIPLIER = 0.3;
+ static final double LAG_AMPLIFICATION_MULTIPLIER = 0.0;
+
+ /**
+ * Amplification multiplier used once aggregate lag crosses {@link
#CRITICAL_LAG_TIER1_FRACTION} of
+ * {@link CostBasedAutoScalerConfig#getCriticalLagThreshold()} (tier 1 of
the critical-lag fast path).
+ */
+ static final double CRITICAL_LAG_AMPLIFICATION_MULTIPLIER = 6.0;
Review Comment:
Let's make this configurable, as discussed offline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
