[
https://issues.apache.org/jira/browse/FLINK-23620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17445114#comment-17445114
]
Mika Naylor commented on FLINK-23620:
-------------------------------------
I have opened a PR for this (https://github.com/apache/flink/pull/17821) -
however, I would appreciate some input whether the following changes make sense
as I forsee 2 problems in the implementation:
- Previously, the yaml parser would just discard rows in the configuration it
percieved as invalid. A configuration file containing some invalid yaml
statements would discard those values, but keep valid ones. This change parses
the entire file as a single yaml document, so any invalid yaml leads the whole
file to be discarded.
- The nested key structure doesn't quite work for everything. For example, the
following configuration would not be able to be written in a nested key format:
{noformat}
high-availability: zookeeper
high-availability.storageDir: hdfs://flink/ha
high-availability.zookeeper.quorum: localhost:2181
high-availability.zookeeper.client.acl: open
{noformat}
as
{noformat}
high-availability: zookeeper
high-avilability:
storageDir: hdfs://flink/ha
zookeeper:
quorum: localhost:2181
client.acl: open
{noformat}
Would mean that the {noformat}high-availability: zookeeper{noformat} key would
be overwritten.
> Introduce proper YAML parsing to Flink's configuration
> ------------------------------------------------------
>
> Key: FLINK-23620
> URL: https://issues.apache.org/jira/browse/FLINK-23620
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
> Reporter: Mika Naylor
> Assignee: Mika Naylor
> Priority: Minor
> Labels: pull-request-available
>
> At the moment, the YAML parsing for Flink's configuration file
> ({{conf/flink-conf.yaml)}} is pretty basic. It only supports basic key value
> pairs, such as:
> {code:java}
> a.b.c: a value
> a.b.d: another value{code}
> As well as supporting some invalid YAML syntax, such as:
> {code:java}
> a: b: value{code}
>
> Introducing proper YAML parsing to the configuration component would let
> Flink users use features such as nested keys, such as:
> {code:java}
> a:
> b:
> c: a value
> d: another value{code}
> as well as make it easier to integrate configuration tools/languages that
> compile to YAML, such as Dhall.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
