[
https://issues.apache.org/jira/browse/FLINK-35746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18008082#comment-18008082
]
Nishita Pattanayak commented on FLINK-35746:
--------------------------------------------
[~gyfora] I had a few questions on the above.
>> *Things that worked out:*
I was able to validate that the programmatic config overrides does end up in
the job's config REST api i.e `jobs/:jobId/checkpoints/config` for checkpoint
related configs and `jobs/:jobId/config` for job specific programmatic configs.
We have respective `CheckpointConfigInfo` and `JobConfigInfo` defined in
`apache/flink` which implements the responseBody and thus allows us to make
REST calls to the above api's to get the user configurations.
I was also able to implement the logic to cache this runtime config to avoid
repeated REST API calls.
>> *Queries*
1. However , I was a little concerned about the specifics of which configs
should be overridden and which should not.
Would it be better to take all the configs defined programmatically as the
source of truth as it is mostly the case. Please do correct me If I am missing
something.
2. I see the names that are being used in response config keys is also
different from what operator expects hence that would also have to be mapped
correctly to ensure operator takes the config correctly.
> Add logic to override observed config based on settings observed through REST
> API
> ---------------------------------------------------------------------------------
>
> Key: FLINK-35746
> URL: https://issues.apache.org/jira/browse/FLINK-35746
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Reporter: Gyula Fora
> Assignee: Nishita Pattanayak
> Priority: Major
>
> In many cases the Flink operator relies on the user configuration to infer
> running application/cluster settings. While this is mostly fine, many of
> these configs can be programmatically changed by the user in their app main
> method that will ultimately take precedence and can lead to inconsistent
> behaviour from the operators side.
> To alleviate this we need to add logic to override parts of the observed
> config based on what we see from the cluster. Such as checkpointing enabled /
> disabled, checkpoint intervals etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
