tweise commented on a change in pull request #13116:
URL: https://github.com/apache/beam/pull/13116#discussion_r506988274
##########
File path:
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkExecutionEnvironments.java
##########
@@ -259,28 +278,44 @@ static StreamExecutionEnvironment
createStreamExecutionEnvironment(
options.setShutdownSourcesAfterIdleMs(0L);
}
}
+ }
- applyLatencyTrackingInterval(flinkStreamEnv.getConfig(), options);
-
- if (options.getAutoWatermarkInterval() != null) {
-
flinkStreamEnv.getConfig().setAutoWatermarkInterval(options.getAutoWatermarkInterval());
- }
-
- // State backend
- if (options.getStateBackendFactory() != null) {
+ private static void configureStateBackend(
Review comment:
Flink is headed in the direction where everything that is set on an
environment becomes configurable (including the executor, FLIP-73). This change
kind of goes in the opposite direction, increasing the amount of Flink pipeline
options further. Should we look into the generic configuration mechanism
instead, where it is really easy for the user to supply the Flink configuration
(optionally inline, instead of via a separate file)?
##########
File path: runners/flink/flink_runner.gradle
##########
@@ -148,6 +148,8 @@ dependencies {
compile "org.apache.flink:flink-java:$flink_version"
compile "org.apache.flink:flink-runtime_2.11:$flink_version"
compile "org.apache.flink:flink-streaming-java_2.11:$flink_version"
+ // RocksDB state backend
+ compile "org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version"
Review comment:
Just wanted to confirm that dependency won't be baked into the job
server as it is already part of the Flink dist.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
