[
https://issues.apache.org/jira/browse/FLINK-38323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-38323:
--------------------------
Description:
As a user, I'd like to have an option to restrict the scaling decisions to
those altering the number of replicas. This could be achieved by using a global
scale factor instead of individually adjusting the parallelism for the
different operators. This way the autoscaler would work as when one simply
adds/removes replicas by hand, which seems to distribute the work more evenly
in some cases.
Along these lines, some users have reported running an internal version of the
autoscaler that makes the whole pipeline (except for the sources/sinks) work
with a global parallelism to avoid uneven task distribution. The change should
be relatively simple, i.e., just run the algorithm per vertex, and then take
the maximum parallelism over all the vertices.
See the question/suggestion first posted
[here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
was:
As a user, I'd like to have an option to restrict to scaling decision altering
the number of replicas. This could be achieved by using a global scale factor
instead of individually adjusting the parallelism for the different operators.
This way the autoscaler would work as when one simply adds/removes replicas by
hand, which seems to distribute the work more evenly in some cases.
Along these lines, some users have reported running an internal version of the
autoscaler that makes the whole pipeline (except for the sources/sinks) work
with a global parallelism to avoid uneven task distribution. The change should
be relatively simple, i.e., just run the algorithm per vertex, and then take
the maximum parallelism over all the vertices.
See the question/suggestion first posted
[here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
> Global Scale Factor Mode
> ------------------------
>
> Key: FLINK-38323
> URL: https://issues.apache.org/jira/browse/FLINK-38323
> Project: Flink
> Issue Type: New Feature
> Components: Autoscaler
> Affects Versions: 1.12.1
> Reporter: Salva
> Priority: Major
> Attachments: TMs - CPU Usage.png
>
>
> As a user, I'd like to have an option to restrict the scaling decisions to
> those altering the number of replicas. This could be achieved by using a
> global scale factor instead of individually adjusting the parallelism for the
> different operators. This way the autoscaler would work as when one simply
> adds/removes replicas by hand, which seems to distribute the work more evenly
> in some cases.
> Along these lines, some users have reported running an internal version of
> the autoscaler that makes the whole pipeline (except for the sources/sinks)
> work with a global parallelism to avoid uneven task distribution. The change
> should be relatively simple, i.e., just run the algorithm per vertex, and
> then take the maximum parallelism over all the vertices.
> See the question/suggestion first posted
> [here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
