laughingman7743 opened a new pull request, #1222:
URL: https://github.com/apache/flink-kubernetes-operator/pull/1222
## What is the purpose of the change
When `spec.taskManager.replicas` is set, the operator ignores
`spec.job.parallelism` and uses `replicas * taskmanager.numberOfTaskSlots`. In
standalone mode the job then occupies every slot, so it cannot keep spare
(standby) TaskManagers to fail over onto when a TaskManager pod is lost. Native
mode covers this with `slotmanager.redundant-taskmanager-num`, which has no
effect in standalone mode. See FLINK-40992 for details.
This pull request adds an opt-out option so that an explicit job parallelism
can take effect while the TaskManager replicas stay as specified. The default
keeps the current behavior.
## Brief change log
- New dynamic option
`kubernetes.operator.job.parallelism.derive-from-replicas` (default `true`).
- `FlinkConfigBuilder#getParallelism()`: when the option is `false` and
`spec.job.parallelism` > 0, the job parallelism is used even if the TaskManager
replicas are set. With parallelism `0` it is still derived from the replicas.
The existing warning now points to the option.
- `DefaultValidator`: in standalone mode with the option set to `false`, a
parallelism larger than the slots of the replicas is rejected, including the
maximum of `pipeline.jobvertex-parallelism-overrides`, since a standalone
cluster cannot add TaskManagers.
- Docs: the Standalone Mode section of the deployment overview, the
validation table in the controller internals, and the regenerated configuration
reference.
- A separate `[hotfix]` commit replaces the only Hamcrest assertion in
`FlinkConfigBuilderTest` with AssertJ, so the new test can use AssertJ.
## Verifying this change
This change added tests and can be verified as follows:
-
`FlinkConfigBuilderTest#testJobParallelismNotDerivedFromTaskManagerReplicas`:
in standalone mode with 3 replicas of 2 slots and a job parallelism of 4, the
parallelism is 6 by default and 4 with the option set to `false`, the
TaskManager replicas stay 3, and a job parallelism of 0 is still derived from
the replicas.
-
`DefaultValidatorTest#testValidateStandaloneParallelismNotDerivedFromReplicas`:
with the option set to `false`, a parallelism or a parallelism override above
the slots of the replicas is rejected in standalone mode; it is accepted within
the slots, with the default option value, and in native mode.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changes to the `CustomResourceDescriptors`:
no
- Core observer or reconciler logic that is regularly executed: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
