[
https://issues.apache.org/jira/browse/FLINK-36640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matyas Orhidi updated FLINK-36640:
----------------------------------
Description:
FLINK-32589 introduced a feature to carry over parallelism overrides between
application upgrades. This serves pretty well on the happy path preventing
users from clearing the actual parallelism settings on regular updates. However
in certain scenarios users may actually want to reset the parallelism to an
initial or desired state to force the application out of an unwanted scaling
state.
Such unwanted state can be for example having too large parallelisms that
causes throttling, or similar. The current workaround needs to upgrade steps by
setting `job.autoscaler.enabled: false` and then `job.autoscaler.enabled: true`
again. We could combine it with a `autoscalerReset` nonce. We could also
reintroduce the config with manual vertex parallelism settings to be able to
reset to a configurable state, similar to what we do when starting a job from
specific Kafka offsets.
was:
FLINK-32589 introduced a feature to carry over parallelism overrides between
application upgrades. This serves pretty well on the happy path preventing
users from clearing the actual parallelism settings on regular updates. However
in certain scenarios users may actually want to reset the parallelism to an
initial or desired state to force the application out of an unwanted scaling
state.
Such unwanted state can be for example too large parallelisms that causes
throttling, or similar. The current workaround needs to upgrade steps by
setting `job.autoscaler.enabled: false` and then `job.autoscaler.enabled: true`
again. We could combine it with a `autoscalerReset` nonce. We could also
reintroduce the config with manual vertex parallelism settings to be able to
reset to a configurable state, similar to what we do when starting a job from
specific Kafka offsets.
> Provide an easy way to reset autoscaling
> ----------------------------------------
>
> Key: FLINK-36640
> URL: https://issues.apache.org/jira/browse/FLINK-36640
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Reporter: Matyas Orhidi
> Priority: Major
>
> FLINK-32589 introduced a feature to carry over parallelism overrides between
> application upgrades. This serves pretty well on the happy path preventing
> users from clearing the actual parallelism settings on regular updates.
> However in certain scenarios users may actually want to reset the parallelism
> to an initial or desired state to force the application out of an unwanted
> scaling state.
> Such unwanted state can be for example having too large parallelisms that
> causes throttling, or similar. The current workaround needs to upgrade steps
> by setting `job.autoscaler.enabled: false` and then `job.autoscaler.enabled:
> true` again. We could combine it with a `autoscalerReset` nonce. We could
> also reintroduce the config with manual vertex parallelism settings to be
> able to reset to a configurable state, similar to what we do when starting a
> job from specific Kafka offsets.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
