[
https://issues.apache.org/jira/browse/FLINK-34504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan updated FLINK-34504:
----------------------------
Description:
JobVertexScaler#scale has a optimization: Try to adjust the parallelism such
that it divides the number of key groups without a remainder => data is evenly
spread across subtasks.
It's only useful when the upstream shuffle type has keyBy. We should avoid this
optimization when the upstream shuffle type doesn't have keyBy.
was:
JobVertexScaler#scale has a optimization: Try to adjust the parallelism such
that it divides the number of key groups without a remainder => data is evenly
spread across subtasks.
It's only useful when the upstream shuffle type has keyBy. We should avoid this
optimization when the upstream shuffle type doesn't has keyBy.
> Avoid the parallelism adjustment when the upstream shuffle type doesn't has
> keyBy
> ---------------------------------------------------------------------------------
>
> Key: FLINK-34504
> URL: https://issues.apache.org/jira/browse/FLINK-34504
> Project: Flink
> Issue Type: Improvement
> Components: Autoscaler
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
> Fix For: kubernetes-operator-1.8.0
>
>
> JobVertexScaler#scale has a optimization: Try to adjust the parallelism such
> that it divides the number of key groups without a remainder => data is
> evenly spread across subtasks.
>
> It's only useful when the upstream shuffle type has keyBy. We should avoid
> this optimization when the upstream shuffle type doesn't have keyBy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
