Dennis-Mircea opened a new pull request, #1178:
URL: https://github.com/apache/flink-kubernetes-operator/pull/1178
## What is the purpose of the change
Follow-up to FLINK-35746, which added runtime configuration observation
(reading a job's effective settings from the Flink REST API and layering them
over the observed configuration).
Two gaps are addressed. The operator documentation still describes the
previous configuration model, to the point of stating the operator cannot
observe the running configuration. And the global job parameters taken from the
REST response are applied unfiltered, so a job can override operator
configuration for its own resource.
## Brief change log
**Documentation** (`docs/content{,.zh}/docs/internals/startup.md`)
- The Configuration section now lists four configurations rather than three,
with a Runtime row describing what it is read from (the JobManager
configuration, job execution and checkpoint config REST endpoints) and what it
is used for.
- The Observe row states that the runtime configuration is layered on top,
matching `FlinkResourceContext#getObserveConfig`.
- A short paragraph explains why the runtime configuration exists, since a
spec is a request rather than a record of what the job ended up running with,
followed by its lifecycle: fetched once per job, skipped for globally terminal
jobs, cached per resource and job id, and falling back to spec-derived values
while a fetch keeps failing.
- The closing warning previously said "None of these is the running
configuration", which FLINK-35746 made untrue. It is replaced rather than
removed, since the mapping only covers a subset of settings and only while the
job runs, so a `config.yaml` baked into the image is still invisible to the
operator.
**Filtering** (`FlinkRuntimeConfigurationUtils`)
- Global job parameters whose key is in the operator's own namespaces
(`kubernetes.operator.` and `job.autoscaler.`) are no longer copied into the
runtime configuration, and each dropped key is logged at WARN.
- The check uses the existing
`KubernetesOperatorConfigOptions.K8S_OP_CONF_PREFIX` and
`AutoScalerOptions.AUTOSCALER_CONF_PREFIX` constants so it follows the prefixes
if they ever change.
The filter is deliberately limited to those two namespaces. The JobManager
configuration endpoint legitimately contributes keys such as `rest.*`, so
filtering more broadly by key name would risk dropping genuinely observed
settings, whereas operator and autoscaler keys have no reason to arrive from a
job's global parameters at all.
## Verifying this change
-
`AbstractFlinkServiceTest#testMapJobConfigurationDropsOperatorControlledGlobalParameters`
asserts that an operator key and an autoscaler key set as global job
parameters are dropped, while an unrelated parameter and the mapped execution
fields are kept.
- The existing `testMapJobConfigurationMapsAllExpectedFields` continues to
pass, so ordinary parameters are unaffected.
- The full `flink-kubernetes-operator` suite passes (2240 tests).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changes to the `CustomResourceDescriptors`: no
- Core observer or reconciler logic that is regularly executed: yes, the
runtime configuration mapping used by the job status observer
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? the Internals startup
documentation is corrected to describe the runtime configuration added in
FLINK-35746
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please
change the checkbox below to `[X]` followed by the name of the tool, and
uncomment the
"Generated-by" line. See the ASF Generative Tooling Guidance for details:
https://www.apache.org/legal/generative-tooling.html
You are responsible for the quality and correctness of every change in this
PR
regardless of the tooling used. Low-effort AI-generated PRs will be closed.
See
AGENTS.md for the full guidance.
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name and Version]
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
