Myracle opened a new pull request, #27523:
URL: https://github.com/apache/flink/pull/27523
## What is the purpose of the change
This pull request adds native support for mounting Kubernetes
PersistentVolumeClaims (PVCs) to Flink JobManager and TaskManager pods through
simple configuration options, eliminating the need to use verbose pod templates
for common PVC mounting scenarios.
Currently, users who want to mount PVCs must use pod templates, which can be
error-prone and verbose. This PR introduces a simpler approach:
`kubernetes.persistent-volume-claims:
checkpoint-pvc:/opt/flink/checkpoints,data-pvc:/opt/flink/data
kubernetes.persistent-volume-claims.read-only: false
`
## Brief change log
- Add kubernetes.persistent-volume-claims configuration option (Map type) to
specify PVCs and their mount paths in format pvc-name:/mount/path
- Add kubernetes.persistent-volume-claims.read-only configuration option
(Boolean, default: false) to control read-only mounting
- Add PersistentVolumeClaimMountDecorator to decorate JobManager and
TaskManager pods with PVC volumes and volume mounts
- Add input validation for PVC names (DNS-1123 subdomain standard, max 253
chars) and mount paths (must be absolute, no duplicates)
- Add volume name conflict detection with fail-fast behavior to prevent K8s
API rejections
- Implement volume name sanitization: replace . with - and truncate with
hash suffix for long names (>63 chars)
- Add getPersistentVolumeClaimsToMountPaths() and
isPersistentVolumeClaimReadOnly() methods to AbstractKubernetesParameters
- Register the decorator in KubernetesJobManagerFactory and
KubernetesTaskManagerFactory
## Verifying this change
This change added tests and can be verified as follows:
- Added PersistentVolumeClaimMountDecoratorTest with 28 test cases covering.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no )
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes, affects
Kubernetes pod deployment)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (docs / JavaDocs)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
