[
https://issues.apache.org/jira/browse/FLINK-38770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18133832#comment-18133832
]
Gerk Elznik commented on FLINK-38770:
-------------------------------------
We hit this on Flink 2.2.1 (Application Mode, adaptive scheduler) with
Kubernetes Operator 1.16.1
autoscaling a 16-vertex streaming job, and can add two observations on how it
interacts with the operator.
1. The override is loaded but has no effect. At startup the JobManager logs the
pipeline.jobvertex-parallelism-overrides map it read from the configuration,
then computes
numberOfRequiredSlots from parallelism.default and deploys every task at
that parallelism.
2. The operator then considers the deployment reconciled, because the override
map in the spec equals
lastReconciledSpec, so it does not trigger an in-place rescale. The job
stays at the default parallelism
until a recommendation differs from the stored map. With a stable workload
that never happens: the
autoscaler emits the same ScalingReport every scale-down interval, the
scalings counter increments,
and nothing changes, so autoscaling is silently non-functional after any job
upgrade or restart.
With a varying workload it recovers once a differing recommendation appears,
through the REST
resource-requirements path ("In-place scaling triggered"), which works fine
on 2.2.1. Only the
config-level application at submission is broken.
A 2.2.x or 2.3.x backport would matter to operator users, since the operator
autoscaler is the main
consumer of this option. We can validate a patched 2.2.x build against this job
in a non-production
environment.
Workaround for the stuck case: changing spec.job.autoscalerResetNonce clears
the stored overrides, so the
next recommendation differs and is applied in place. It only lasts until the
next restart.
> pipeline.jobvertex-parallelism-overrides ignored for StreamGraph in
> Application Mode
> ------------------------------------------------------------------------------------
>
> Key: FLINK-38770
> URL: https://issues.apache.org/jira/browse/FLINK-38770
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 2.0.0, 2.1.0, 2.0.1, 2.2.0, 2.1.1, 2.3.0
> Reporter: Barak
> Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
