[
https://issues.apache.org/jira/browse/FLINK-38323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-38323:
--------------------------
Description:
As a user, I'd like to have an option to restrict to scaling decision altering
the number of replicas. This could be achieved by using a global scale factor
instead of individually adjusting the parallelism for the different operators.
This way the autoscaler would work as when one simply adds/removes replicas by
hand. This seems to more evenly distribute the work in some cases.
Along these lines, some users have reported running an internal version of the
autoscaler that makes the whole pipeline (except for the sources/sinks) work
with a global parallelism avoid uneven task distribution. The change should be
relatively simple, i.e., just run the algorithm per vertex, and take the max of
them.
See the question/suggestion first posted
[here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
was:
As a user, I'd like to have an option to restrict to scaling decision altering
the number of replicas. This could be achieved by using a global scale factor
instead of individually adjusting the parallelism for the different operators.
This would make the autoscaler work as when one adds/removes replicas by hand.
Some users have reported running an internal version of the autoscaler with the
one major difference being that they let the whole pipeline (except for the
sources/sinks) to have the same parallelism to avoid uneven task distribution.
The change should be relatively simple, i.e., just run the algorithm per
vertex, and take the max of them.
See the question/suggestion first posted
[here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
> Global Scale Factor Mode
> ------------------------
>
> Key: FLINK-38323
> URL: https://issues.apache.org/jira/browse/FLINK-38323
> Project: Flink
> Issue Type: New Feature
> Components: Autoscaler
> Affects Versions: 1.12.1
> Reporter: Salva
> Priority: Major
> Attachments: TMs - CPU Usage.png
>
>
> As a user, I'd like to have an option to restrict to scaling decision
> altering the number of replicas. This could be achieved by using a global
> scale factor instead of individually adjusting the parallelism for the
> different operators. This way the autoscaler would work as when one simply
> adds/removes replicas by hand. This seems to more evenly distribute the work
> in some cases.
> Along these lines, some users have reported running an internal version of
> the autoscaler that makes the whole pipeline (except for the sources/sinks)
> work with a global parallelism avoid uneven task distribution. The change
> should be relatively simple, i.e., just run the algorithm per vertex, and
> take the max of them.
> See the question/suggestion first posted
> [here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
