[
https://issues.apache.org/jira/browse/FLINK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131014#comment-18131014
]
yongfu.gao commented on FLINK-40873:
------------------------------------
I double checked all operator config options: scanned 159 ConfigOptions across
flink-kubernetes-operator, flink-autoscaler, flink-autoscaler-standalone and
flink-autoscaler-plugin-jdbc. Exactly three keys are a strict prefix of another
key
(dynamic <name> templates are covered too):
job.autoscaler.scaling.parallelism-alignment.mode ->
...mode.<name>.\{class,<parameter>}
kubernetes.operator.savepoint.history.max.age -> ...max.age.threshold
kubernetes.operator.savepoint.history.max.count -> ...max.count.threshold
No other key overlaps. Note the second and third conflict with their .threshold
variants, not with each other.
Confirmed the failure with flink-core 1.20.1:
YamlParserUtils#convertAndDumpYamlFromFlatMap
fails with "ClassCastException: String cannot be cast to Map" whenever both
sides of
one pair are present.
Proposed fix (keep the older, documented leaves; rename the newer side):
1. ...alignment.mode.<name>.\{class,<parameter>} ->
...alignment.modes.<name>.\{class,<parameter>}
(leaf unchanged; template fallback read with a deprecation warning)
2. savepoint.history.max.age.threshold ->
savepoint.history.threshold.max.age
3. savepoint.history.max.count.threshold ->
savepoint.history.threshold.max.count
(both static keys, withDeprecatedKeys keeps old configs working)
If you prefer renaming the leaves instead: ...alignment.mode ->
...alignment.active-mode
(withDeprecatedKeys), thresholds stay as they are. My preference is the first
variant.
The PR would also add a unit test that reflects over all ConfigOptions and
fails when one
key is a strict prefix of another, plus a YAML round-trip test reproducing the
exception.
Happy to open it once you agree on the naming.
> Fix overlapping config key prefixes in the operator
> ---------------------------------------------------
>
> Key: FLINK-40873
> URL: https://issues.apache.org/jira/browse/FLINK-40873
> Project: Flink
> Issue Type: Bug
> Components: Kubernetes Operator
> Reporter: Gyula Fora
> Priority: Major
>
> To properly support yaml configs, we need to fix and remove overlapping
> config key prefixes.
> Some problematic ones:
> - job.autoscaler.scaling.parallelism-alignment.mode
> - kubernetes.operator.savepoint.history.max.age
> - kubernetes.operator.savepoint.history.max.count
>
> All other keys must be double checked
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
