Re: Enable RocksDB in FlinkDeployment with flink-kubernetes-operator

2023-08-30 Thread Gyula Fóra
I agree with Yaroslav, generally speaking PVs are not necessary or even recommended for RocksDB because the state doesn't need to be shared, recovered later anyways. It's usually faster and cheaper to go with instance level SSDs. Gyula On Wed, Aug 30, 2023 at 8:37 PM Yaroslav Tkachenko wrote: >

Re: Enable RocksDB in FlinkDeployment with flink-kubernetes-operator

2023-08-30 Thread Yaroslav Tkachenko
It depends on your requirements. Personally, I don't use PVs and, instead, mount a volume from a host with a fast instance-level SSD. On Wed, Aug 30, 2023 at 11:26 AM Tony Chen wrote: > We used to have a Persistent Volume (PV), attached to the pod, for storing > the RocksDB data while using the

Re: Enable RocksDB in FlinkDeployment with flink-kubernetes-operator

2023-08-30 Thread Tony Chen
We used to have a Persistent Volume (PV), attached to the pod, for storing the RocksDB data while using the GoogleCloudPlatform operator. For the Apache flink-kubernetes-operator, do the pods need a PV attached to it to use RocksDB? If not, do you have recommendations on memory configuration for th

Re: Enable RocksDB in FlinkDeployment with flink-kubernetes-operator

2023-08-30 Thread Gyula Fóra
Hi! Rocksdb is supported and every other state backend as well. You can simply set this in you config like before :) Cheers Gyula On Wed, 30 Aug 2023 at 19:22, Tony Chen wrote: > Hi Flink Community, > > Does the flink-kubernetes-operator support RocksDB as the state backend > for FlinkDeploym

Re: Enable RocksDB in FlinkDeployment with flink-kubernetes-operator

2023-08-30 Thread Yaroslav Tkachenko
Hey Tony, Pretty much all Flink configuration is supported, including the RocksDB state backend. On Wed, Aug 30, 2023 at 9:05 AM Tony Chen wrote: > Hi Flink Community, > > Does the flink-kubernetes-operator support RocksDB as the state backend > for FlinkDeployment? > > We have some Flink appli

Enable RocksDB in FlinkDeployment with flink-kubernetes-operator

2023-08-30 Thread Tony Chen
Hi Flink Community, Does the flink-kubernetes-operator support RocksDB as the state backend for FlinkDeployment? We have some Flink applications that have large states, and we were able to deal with these large states in the past with RocksDB. If there is no support for RocksDB, are there any rec