[
https://issues.apache.org/jira/browse/FLINK-31608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl closed FLINK-31608.
---------------------------------
Resolution: Invalid
This topic was covered by
[FLIP-472|https://cwiki.apache.org/confluence/display/FLINK/FLIP-472%3A+Aligning+timeout+logic+in+the+AdaptiveScheduler's+WaitingForResources+and+Executing+states]
> Re-evaluate 'min-parallelism-increase' option
> ---------------------------------------------
>
> Key: FLINK-31608
> URL: https://issues.apache.org/jira/browse/FLINK-31608
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration, Runtime / Coordination
> Reporter: Chesnay Schepler
> Priority: Major
> Fix For: 2.0.0
>
>
> This option was meant to prevent scale up operations where the benefit
> doesn't outweigh the cost, like scaling up to increase a single vertices
> parallelism by 1. Meanwhile, scale-down operations were always immediately
> executed, because they were always the result of a stopped TaskManager,
> causing the job to restart anyway.
> Now that users can change the requirements at will this has changed, and the
> expected behavior is overall undefined.
> We need to answer:
> * should there be a dedicated option for limiting scale-down operations if
> the requirements were changed?
> * should the min-parallelism-{*}increase{*} option be generalized to a
> min-parallelism-{*}change{*} option?
> * How shall operations be handled that scale different vertices up or down
> at the same? So far the decision was made on the cumulative parallelism
> change, but in this case the parallelism distribution can change
> significantly while the cumulative change is 0.
> * If a rescale operation was not applied due to these limits, should they be
> _eventually_ applied anyway (e.g., after a timeout)?
>
> See [https://github.com/apache/flink/pull/22169#discussion_r1147447567] for
> the background.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
