Kezhu Wang created FLINK-14790:
----------------------------------
Summary: Custom state backend in job graph composition without
depending on concrete state backend implementation
Key: FLINK-14790
URL: https://issues.apache.org/jira/browse/FLINK-14790
Project: Flink
Issue Type: New Feature
Components: API / DataStream
Reporter: Kezhu Wang
In Flink 1.9 and before, if we want to custom state backend, we have to supply
concrete state backend implementation in
{{StreamExecutionEnvironment.setStateBackend}}. This implies that we have to
depend on that concrete state backend implementation in job graph composition
phase since we have to construct this concrete state backend. This may not be
appropriate if we separate job graph composition and execution strictly, as job
graph composition requires no state backend functions besides customization.
Suppose that we are going to build a declarative streaming platform that
building plain json/yaml like textual object, which may contain custom state
backend configuration, with attached dependent jars to job graph and submitting
generated job graph to Flink, we apparently should avoid ad-hoc state backend
construction due to code reusability and indeterministic state backends
configured.
So I propose to state backend customization in job graph composition phase to
not depend on concrete state backend implementation.
To achieve this goal, a {{Configuration}} field, say
{{StreamExecutionEnvironment.stateBackendConfiguration}}, should be sufficient.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
