[
https://issues.apache.org/jira/browse/FLINK-30530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arseniy Tashoyan updated FLINK-30530:
-------------------------------------
Description:
Currently the Flink configuration can be specified in the YAML descriptor of
FlinkDeployment via the _flinkConfiguration_ setting:
{code:yaml}
flinkConfiguration:
taskmanager.numberOfTaskSlots: "2"
...
{code}
Same for the logging configuration:
{code:yaml}
logConfiguration:
"log4j-console.properties": |
rootLogger.level = DEBUG
...{code}
This makes the YAML descriptor overloaded and huge. In addition, Flink and
logging configuration may differ for different applications, while the
Kubernetes settings maybe same for all applications. Therefore it makes sense
to extract Flink and logging configurations from the YAML descriptor.
This can be done via a user-provided ConfigMap:
{code:yaml}
flinkConfigMap: basic-example-flink-config
{code}
In this example we have a Flink application {_}basic-example{_}. The
_basic-example-flink-config_ ConfigMap contains all config files used by Flink:
flink-conf.yaml, log4j-console.properties, possibly other files. The content of
this ConfigMap gets mounted as a volume to {_}/opt/flink/conf{_}.
Therefore we can have different Flink settings for different applications and
the same YAML descriptor for all of them (only the value for _flinkConfigMap_
differs).
was:
Currently the Flink configuration can be specified in the YAML descriptor of
FlinkDeployment via the _flinkConfiguration_ setting:
{code:yaml}
flinkConfiguration:
taskmanager.numberOfTaskSlots: "2"
...
{code}
Same for the logging configuration:
{code:yaml}
logConfiguration:
"log4j-console.properties": |
rootLogger.level = DEBUG
...{code}
This makes the YAML descriptor overloaded and huge. In addition, Flink and
logging configuration may differ for different applications, while the
Kubernetes settings maybe same for all applications. Therefore it makes sense
to extract Flink and logging configurations from the YAML descriptor.
This can be done via a user-provided ConfigMap:
{code:yaml}
flinkConfigMap: basic-example-flink-config
{code}
In this example we have a Flink application {_}basic-example{_}. The
_basic-example-flink-config_ ConfigMap contains all config files used by Flink:
flink-conf.yaml, log4j-console.properties, possibly other files.
Therefore we can have different Flink settings for different applications and
the same YAML descriptor for all of them (only the value for _flinkConfigMap_
differs).
> Flink configuration from user-provided ConfigMap
> ------------------------------------------------
>
> Key: FLINK-30530
> URL: https://issues.apache.org/jira/browse/FLINK-30530
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Affects Versions: 1.15.2
> Environment: Flink 1.15.2
> Flink Kubernetes operator 1.2.0
> Reporter: Arseniy Tashoyan
> Priority: Major
>
> Currently the Flink configuration can be specified in the YAML descriptor of
> FlinkDeployment via the _flinkConfiguration_ setting:
> {code:yaml}
> flinkConfiguration:
> taskmanager.numberOfTaskSlots: "2"
> ...
> {code}
> Same for the logging configuration:
> {code:yaml}
> logConfiguration:
> "log4j-console.properties": |
> rootLogger.level = DEBUG
> ...{code}
> This makes the YAML descriptor overloaded and huge. In addition, Flink and
> logging configuration may differ for different applications, while the
> Kubernetes settings maybe same for all applications. Therefore it makes sense
> to extract Flink and logging configurations from the YAML descriptor.
> This can be done via a user-provided ConfigMap:
> {code:yaml}
> flinkConfigMap: basic-example-flink-config
> {code}
> In this example we have a Flink application {_}basic-example{_}. The
> _basic-example-flink-config_ ConfigMap contains all config files used by
> Flink: flink-conf.yaml, log4j-console.properties, possibly other files. The
> content of this ConfigMap gets mounted as a volume to {_}/opt/flink/conf{_}.
> Therefore we can have different Flink settings for different applications and
> the same YAML descriptor for all of them (only the value for _flinkConfigMap_
> differs).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
