ChenlingXie opened a new pull request, #28693:
URL: https://github.com/apache/flink/pull/28693
## What is the purpose of the change
This pull request allows Kubernetes deployments to configure memory requests
independently from memory limits for JobManager and TaskManager containers.
Previously, Kubernetes memory requests were always set to the configured
process memory, while memory limits could be adjusted with the existing memory
limit factor. This change adds request factors so users can lower Kubernetes
memory requests while preserving the existing limit-factor behavior.
## Brief change log
- Added `kubernetes.jobmanager.memory.request-factor` and
`kubernetes.taskmanager.memory.request-factor`, both defaulting to `1.0`.
- Applied the memory request factor when generating JobManager and
TaskManager Kubernetes resource requirements.
- Added validation so memory request factors must be greater than `0` and
less than or equal to `1`.
- Added unit tests for parameter validation and generated resource
requirements.
- Regenerated the Kubernetes configuration documentation.
## Verifying this change
This change added tests and can be verified as follows:
- Added coverage in `InitJobManagerDecoratorTest` and
`InitTaskManagerDecoratorTest` for generated memory requests while memory
limits still use the existing limit factors.
- Added coverage in `KubernetesJobManagerParametersTest` and
`KubernetesTaskManagerParametersTest` for default, custom, and invalid memory
request factors.
- Ran:
`mvn -pl flink-kubernetes -am
"-Dtest=InitJobManagerDecoratorTest,InitTaskManagerDecoratorTest,KubernetesJobManagerParametersTest,KubernetesTaskManagerParametersTest"
test -DskipITs "-Drat.skip=true" "-Dcheckstyle.skip=true"
"-DfailIfNoTests=false" "-Dsurefire.failIfNoSpecifiedTests=false"`
- Ran:
`mvn package -Dgenerate-config-docs -pl flink-docs -am -nsu -DskipTests
-DskipITs -Dfast "-Drat.skip=true" "-Dcheckstyle.skip=true" -Pskip-webui-build`
- Ran `git diff --check`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes, Kubernetes
resource requirements
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (OpenAI Codex)
Generated-by: OpenAI Codex
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
