Hi Cam I think FLINK-7289 [1] might offer you some insights to control RocksDB memory, especially the idea using write buffer manager [2] to control the total write buffer memory. If you do not have too many sst files, write buffer memory usage would consume much more space than index and filter usage. Since Flink would use per state per column family, and the write buffer number increase when more column families created.
[1] https://issues.apache.org/jira/browse/FLINK-7289 [2] https://github.com/dataArtisans/frocksdb/pull/4 Best Yun Tang ________________________________ From: Cam Mach <cammac...@gmail.com> Sent: Thursday, August 8, 2019 21:39 To: Biao Liu <mmyy1...@gmail.com> Cc: miki haiat <miko5...@gmail.com>; user <user@flink.apache.org> Subject: Re: Capping RocksDb memory usage Thanks for your response, Biao. On Wed, Aug 7, 2019 at 11:41 PM Biao Liu <mmyy1...@gmail.com<mailto:mmyy1...@gmail.com>> wrote: Hi Cam, AFAIK, that's not an easy thing. Actually it's more like a Rocksdb issue. There is a document explaining the memory usage of Rocksdb [1]. It might be helpful. You could define your own option to tune Rocksdb through "state.backend.rocksdb.options-factory" [2]. However I would suggest not to do this unless you are fully experienced of Rocksdb. IMO it's quite complicated. Meanwhile I can share a bit experience of this. We have tried to put the cache and filter into block cache before. It's useful to control the memory usage. But the performance might be affected at the same time. Anyway you could try and tune it. Good luck! 1. https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB 2. https://ci.apache.org/projects/flink/flink-docs-master/ops/state/large_state_tuning.html#tuning-rocksdb Thanks, Biao /'bɪ.aʊ/ On Thu, Aug 8, 2019 at 11:44 AM Cam Mach <cammac...@gmail.com<mailto:cammac...@gmail.com>> wrote: Yes, that is correct. Cam Mach Software Engineer E-mail: cammac...@gmail.com<mailto:cammac...@gmail.com> Tel: 206 972 2768 On Wed, Aug 7, 2019 at 8:33 PM Biao Liu <mmyy1...@gmail.com<mailto:mmyy1...@gmail.com>> wrote: Hi Cam, Do you mean you want to limit the memory usage of RocksDB state backend? Thanks, Biao /'bɪ.aʊ/ On Thu, Aug 8, 2019 at 2:23 AM miki haiat <miko5...@gmail.com<mailto:miko5...@gmail.com>> wrote: I think using metrics exporter is the easiest way [1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#rocksdb On Wed, Aug 7, 2019, 20:28 Cam Mach <cammac...@gmail.com<mailto:cammac...@gmail.com>> wrote: Hello everyone, What is the most easy and efficiently way to cap RocksDb's memory usage? Thanks, Cam
