[ https://issues.apache.org/jira/browse/KAFKA-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damian Guy reassigned KAFKA-4484: --------------------------------- Assignee: Damian Guy > Set more conservative default values on RocksDB for memory usage > ---------------------------------------------------------------- > > Key: KAFKA-4484 > URL: https://issues.apache.org/jira/browse/KAFKA-4484 > Project: Kafka > Issue Type: Bug > Components: streams > Reporter: Guozhang Wang > Assignee: Damian Guy > Labels: performance > > Quoting from email thread: > {code} > The block cache size defaults to a whopping 100Mb per store, and that gets > expensive > fast. I reduced it to a few megabytes. My data size is so big that I doubt > it is very effective > anyway. Now it seems more stable. > I'd say that a smaller default makes sense, especially because the failure > case is > so opaque (running all tests just fine but with a serious dataset it dies > slowly) > {code} > {code} > Before we have the a single-knob memory management feature, I'd like to > propose reducing the Streams' default config values for RocksDB caching and > memory block size. For example, I remember Henry has done some fine tuning on > the RocksDB config for his use case: > https://github.com/HenryCaiHaiying/kafka/commit/b297f7c585f5a883ee068277e5f0f1224c347bd4 > https://github.com/HenryCaiHaiying/kafka/commit/eed1726d16e528d813755a6e66b49d0bf14e8803 > https://github.com/HenryCaiHaiying/kafka/commit/ccc4e25b110cd33eea47b40a2f6bf17ba0924576 > We could check if some of those changes are appropriate in general and if yes > change the default settings accordingly. > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)