gyfora commented on code in PR #586:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/586#discussion_r1584144759
##########
flink-kubernetes-operator-autoscaler/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/config/AutoScalerOptions.java:
##########
@@ -68,15 +68,16 @@ private static ConfigOptions.OptionBuilder
autoScalerConfig(String key) {
public static final ConfigOption<Double> TARGET_UTILIZATION_BOUNDARY =
autoScalerConfig("target.utilization.boundary")
.doubleType()
- .defaultValue(0.1)
+ .defaultValue(0.4)
Review Comment:
For Kafka sources the current lag is taken into account as well. Source
parallelism is adjusted mostly based on 3 different things:
1. Incoming data rate (how much new data comes in per second to the topic)
2. Catch up duration -> catch up extra data rate (current_lag /
catchup_duration_seconds)
3. True processing rate (processing capacity of the source)
If ([1] + [2]) > [3], then we scale up, otherwise we don't. From this you
can see that the shorter the catchup duration config the higher [2] will be
that is the extra capacity required to process the backlog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
