I agree with Yaroslav, generally speaking PVs are not necessary or even recommended for RocksDB because the state doesn't need to be shared, recovered later anyways. It's usually faster and cheaper to go with instance level SSDs.
Gyula On Wed, Aug 30, 2023 at 8:37 PM Yaroslav Tkachenko <yaros...@goldsky.com> wrote: > It depends on your requirements. Personally, I don't use PVs and, instead, > mount a volume from a host with a fast instance-level SSD. > > On Wed, Aug 30, 2023 at 11:26 AM Tony Chen <tony.ch...@robinhood.com> > wrote: > >> We used to have a Persistent Volume (PV), attached to the pod, for >> storing the RocksDB data while using the GoogleCloudPlatform operator. For >> the Apache flink-kubernetes-operator, do the pods need a PV attached to it >> to use RocksDB? If not, do you have recommendations on memory configuration >> for these pods? >> >> I will also need to go through the documentation more on memory >> configuration: >> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/state/state_backends/ >> >> On Wed, Aug 30, 2023 at 2:17 PM Gyula Fóra <gyula.f...@gmail.com> wrote: >> >>> Hi! >>> >>> Rocksdb is supported and every other state backend as well. >>> >>> You can simply set this in you config like before :) >>> >>> Cheers >>> Gyula >>> >>> On Wed, 30 Aug 2023 at 19:22, Tony Chen <tony.ch...@robinhood.com> >>> wrote: >>> >>>> Hi Flink Community, >>>> >>>> Does the flink-kubernetes-operator support RocksDB as the state backend >>>> for FlinkDeployment? >>>> >>>> We have some Flink applications that have large states, and we were >>>> able to deal with these large states in the past with RocksDB. If there is >>>> no support for RocksDB, are there any recommendations on how we can >>>> decrease the size of these states? >>>> >>>> Thanks, >>>> Tony >>>> >>>> >>>> -- >>>> >>>> <http://www.robinhood.com/> >>>> >>>> Tony Chen >>>> >>>> Software Engineer >>>> >>>> Menlo Park, CA >>>> >>>> Don't copy, share, or use this email without permission. If you >>>> received it by accident, please let us know and then delete it right away. >>>> >>> >> >> -- >> >> <http://www.robinhood.com/> >> >> Tony Chen >> >> Software Engineer >> >> Menlo Park, CA >> >> Don't copy, share, or use this email without permission. If you received >> it by accident, please let us know and then delete it right away. >> >
