Re: Running Nexmark for Flink Streaming

2020-04-29 Thread Maximilian Michels
@Sruthi: You'll have to implement your own factory. If you include the mentioned dependency, e.g. if you use Gradle: runtimeOnly "org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version" Then create the factory: options.setStateBackend(o -> new RocksDBStateBackend("file://...")); Al

Re: Running Nexmark for Flink Streaming

2020-04-28 Thread Ismaël Mejía
Max would it make sense to make the rocksdb runtime only at the runner level just to hint its use. I assume that most Flink users might want to have RocksDB as the default state backend? runtimeOnly "org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version" On Tue, Apr 28, 2020 at 1:1

Re: Running Nexmark for Flink Streaming

2020-04-28 Thread Sruthi Sree Kumar
Hello, I have looked the Beam code. The statebackend that we pass should be an instance of FlinkStateBackendFactory . But there is no implementation for the Interface. None of the FlinkStateBackend Implements this interface. So even when I try to pass the default MemoryStateBackend as an argumen

Re: Running Nexmark for Flink Streaming

2020-04-28 Thread Maximilian Michels
Hi Sruthi, Not possible out-of-the-box at the moment. You'll have to add the RocksDB Flink dependency in flink_runner.gradle, e.g.: compile "org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version" Also in the Flink config you have to set state.backend: rocksdb Then you can run Nex

Running Nexmark for Flink Streaming

2020-04-28 Thread Sruthi Sree Kumar
Hello, Is it possible to run the nexmark queries by specifying a state-backed(Ex: RocksDB) ? Regards, Sruthi