Hi everyone,
discussions around ConfigOption seem to be very popular recently. So I
would also like to get some opinions on a different topic.
How do we represent hierarchies in ConfigOption? In FLIP-122, we agreed
on the following DDL syntax:
CREATE TABLE fs_table (
...
) WITH (
'connector' = 'filesystem',
'path' = 'file:///path/to/whatever',
'format' = 'csv',
'format.allow-comments' = 'true',
'format.ignore-parse-errors' = 'true'
);
Of course this is slightly different from regular Flink core
configuration but a connector still needs to be configured based on
these options.
However, I think this FLIP violates our code style guidelines because
'format' = 'json',
'format.fail-on-missing-field' = 'false'
is an invalid hierarchy. `format` cannot be a string and a top-level
object at the same time.
We have similar problems in our runtime configuration:
state.backend=
state.backend.incremental=
restart-strategy=
restart-strategy.fixed-delay.delay=
high-availability=
high-availability.cluster-id=
The code style guide states "Think of the configuration as nested
objects (JSON style)". So such hierarchies cannot be represented in a
nested JSON style.
Therefore, should we advocate instead:
'format.kind' = 'json',
'format.fail-on-missing-field' = 'false'
What do you think?
Thanks,
Timo
[1]
https://flink.apache.org/contributing/code-style-and-quality-components.html#configuration-changes
