Gyula Fora created FLINK-31215:
----------------------------------
Summary: Backpropagate processing rate limits from non-scalable
bottlenecks to upstream operators
Key: FLINK-31215
URL: https://issues.apache.org/jira/browse/FLINK-31215
Project: Flink
Issue Type: New Feature
Components: Autoscaler, Kubernetes Operator
Reporter: Gyula Fora
The current algorithm scales operators based on input data rates by propagating
it forward through the graph.
However there are cases where a certain operators processing capacity is
limited either because it has a set maxParallelism or the users excludes it
from scaling (or otherwise the capacity doesnt increase with scaling).
In these cases it doesn't make sense to scale upstream operators to the target
data rate if the job is going to be bottlenecked by a downstream operator. But
instead we should backpropagate the limit based on the non-scalable bottleneck.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
