Maximilian Michels created BEAM-9855:
----------------------------------------
Summary: Make it easier to configure a Flink state backend
Key: BEAM-9855
URL: https://issues.apache.org/jira/browse/BEAM-9855
Project: Beam
Issue Type: Improvement
Components: runner-flink
Reporter: Maximilian Michels
We should make it easier to configure a Flink state backend. At the moment,
users have to either (1) configure the default state backend in their Flink
cluster, or make sure (2a) they include the dependency in their Gradle/Maven
project (e.g.
{{"org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version"}} for
RocksDB) (2b) set the state backend factory in the {{FlinkPipelineOptions}.
The drawback of option (2) is that it only works in Java due to the factory
specification being in Java.
We can make it easier by simple adding pipeline options for the state backend
name and the checkpoint directory which will be enough for configuring the
state backend. We can add the RocksDB state backend as a default dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
