kfaraz commented on code in PR #19655:
URL: https://github.com/apache/druid/pull/19655#discussion_r3673838385
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/CostBasedAutoScaler.java:
##########
@@ -214,6 +214,25 @@ public CostBasedAutoScalerConfig getConfig()
return config;
}
+ private boolean isCriticalLag(CostMetrics metrics)
+ {
+ final Long criticalLagThreshold = config.getCriticalLagThreshold();
+ return metrics != null && criticalLagThreshold != null
+ && metrics.getAggregateLag() >= criticalLagThreshold *
WeightedCostFunction.CRITICAL_LAG_TIER1_FRACTION;
+ }
+
+ /**
+ * Whether the last collected metrics crossed {@link
WeightedCostFunction#CRITICAL_LAG_TIER2_FRACTION} of
+ * {@link CostBasedAutoScalerConfig#getCriticalLagThreshold()}, meaning the
argmin search should be
+ * skipped entirely in favor of jumping straight to the maximum task count.
+ */
+ private boolean isEmergencyLag(CostMetrics metrics)
Review Comment:
Let's call this state as `critical lag` and the other state as simply `high
lag`. The terms critical and emergency can be confusing, especially since
"emergency lag" currently corresponds to 100% of critical lag.
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/CostBasedAutoScalerConfig.java:
##########
@@ -46,8 +46,9 @@ public class CostBasedAutoScalerConfig implements
AutoScalerConfig
{
static final double DEFAULT_LAG_WEIGHT = 0.4;
static final double DEFAULT_IDLE_WEIGHT = 0.6;
- static final Duration DEFAULT_MIN_SCALE_UP_DELAY =
Duration.standardMinutes(10);
- static final Duration DEFAULT_MIN_SCALE_DOWN_DELAY =
Duration.standardMinutes(30);
+ static final Duration DEFAULT_MIN_SCALE_UP_DELAY =
Duration.standardMinutes(15);
+ static final Duration DEFAULT_MIN_SCALE_DOWN_DELAY =
Duration.standardMinutes(20);
Review Comment:
```suggestion
static final Duration DEFAULT_MIN_SCALE_DOWN_DELAY =
Duration.standardMinutes(30);
```
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/CostBasedAutoScalerConfig.java:
##########
@@ -87,11 +90,13 @@ public CostBasedAutoScalerConfig(
@Nullable @JsonProperty("minScaleDownDelay") Duration minScaleDownDelay,
@Nullable @JsonProperty("scaleDownDuringTaskRolloverOnly") Boolean
scaleDownDuringTaskRolloverOnly,
@Nullable @JsonProperty("usePollIdleRatio") Boolean usePollIdleRatio,
- @Nullable @JsonProperty("minCostDropPercentForScaling") Integer
minCostDropPercentForScaling
+ @Nullable @JsonProperty("criticalLagThreshold") Long
criticalLagThreshold,
+ @Nullable @JsonProperty("minCostDropPercentForScaling") Integer
minCostDropPercentForScaling,
+ @Nullable @JsonProperty("criticalLagAmplificationMultiplier") Double
criticalLagAmplificationMultiplier
Review Comment:
Maybe use a simpler name.
```suggestion
@Nullable @JsonProperty("highLagCostFactor") Double highLagCostFactor
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
