[
https://issues.apache.org/jira/browse/FLINK-38323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Salva updated FLINK-38323:
--------------------------
Description:
As a user, I'd like to have an option to restrict the scaling decisions to
those altering the number of replicas. This could be achieved by using a global
scale factor instead of individually adjusting the parallelism of the different
operators. In the specific operating mode, the autoscaler would work as when
one simply adds/removes replicas by hand, which seems to distribute the work
more evenly in some cases.
Along these lines, some users have reported running an internal version of the
autoscaler that makes the whole pipeline (except for the sources/sinks) work
with a global parallelism to avoid uneven task distribution. The change should
be relatively simple, i.e., just run the algorithm per vertex, and then take
the maximum parallelism over all the vertices.
See the question/suggestion first posted
[here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
was:
As a user, I'd like to have an option to restrict the scaling decisions to
those altering the number of replicas. This could be achieved by using a global
scale factor instead of individually adjusting the parallelism for the
different operators. This way the autoscaler would work as when one simply
adds/removes replicas by hand, which seems to distribute the work more evenly
in some cases.
Along these lines, some users have reported running an internal version of the
autoscaler that makes the whole pipeline (except for the sources/sinks) work
with a global parallelism to avoid uneven task distribution. The change should
be relatively simple, i.e., just run the algorithm per vertex, and then take
the maximum parallelism over all the vertices.
See the question/suggestion first posted
[here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
> Global Scale Factor Mode
> ------------------------
>
> Key: FLINK-38323
> URL: https://issues.apache.org/jira/browse/FLINK-38323
> Project: Flink
> Issue Type: New Feature
> Components: Autoscaler
> Affects Versions: 1.12.1
> Reporter: Salva
> Priority: Major
> Attachments: TMs - CPU Usage.png
>
>
> As a user, I'd like to have an option to restrict the scaling decisions to
> those altering the number of replicas. This could be achieved by using a
> global scale factor instead of individually adjusting the parallelism of the
> different operators. In the specific operating mode, the autoscaler would
> work as when one simply adds/removes replicas by hand, which seems to
> distribute the work more evenly in some cases.
> Along these lines, some users have reported running an internal version of
> the autoscaler that makes the whole pipeline (except for the sources/sinks)
> work with a global parallelism to avoid uneven task distribution. The change
> should be relatively simple, i.e., just run the algorithm per vertex, and
> then take the maximum parallelism over all the vertices.
> See the question/suggestion first posted
> [here|[https://lists.apache.org/thread/rf9fpr6cjnjjwl6rqhtbhfn380doghyl]].
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
