[
https://issues.apache.org/jira/browse/KAFKA-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840812#comment-16840812
]
ASF GitHub Bot commented on KAFKA-8215:
---------------------------------------
ableegoldman commented on pull request #6743: KAFKA-8215: Upgrade Rocks to
v5.18.3
URL: https://github.com/apache/kafka/pull/6743
This upgrade exposes a number of new options, including the
WriteBufferManager which -- along with existing TableConfig options -- allows
users to limit the total memory used by RocksDB across instances. This can
alleviate some cascading OOM potential when, for example, a large number of
stateful tasks are suddenly migrated to the same host.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Limit memory usage of RocksDB
> -----------------------------
>
> Key: KAFKA-8215
> URL: https://issues.apache.org/jira/browse/KAFKA-8215
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Sophie Blee-Goldman
> Priority: Major
>
> The memory usage of Streams is currently unbounded in part because of
> RocksDB, which consumes memory on a per-instance basis. Each instance (ie
> each persistent state store) will have its own write buffer, index blocks,
> and block cache. The size of these can be configured individually, but there
> is currently no way for a Streams app to limit the total memory available
> across instances.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
