Gyula Fora created FLINK-30652:
----------------------------------
Summary: Use max busytime instead of average to compute true
processing rate
Key: FLINK-30652
URL: https://issues.apache.org/jira/browse/FLINK-30652
Project: Flink
Issue Type: Improvement
Components: Autoscaler, Kubernetes Operator
Affects Versions: kubernetes-operator-1.4.0
Reporter: Gyula Fora
Assignee: Gyula Fora
Fix For: kubernetes-operator-1.4.0
Currently we use the some of busyTimes and processed records to estimate the
true processing rate.
This computation however is not correct when any data skew is present as TPR is
not fully additive. The first task to reach 100% utilization will set a limit
to the pipeline processing rate through backpressure.
To avoid this we should use the max busyTime and compute the TPR from that.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
