[
https://issues.apache.org/jira/browse/KAFKA-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772072#comment-17772072
]
Matthias J. Sax commented on KAFKA-15541:
-----------------------------------------
Did not read the KIP yet, but sound very similar to
https://issues.apache.org/jira/browse/KAFKA-15437 – should we close K15437 as
duplicate?
> RocksDB Iterator Metrics
> ------------------------
>
> Key: KAFKA-15541
> URL: https://issues.apache.org/jira/browse/KAFKA-15541
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Nicholas Telford
> Assignee: Nicholas Telford
> Priority: Major
> Labels: kip, kip-required
>
> [KIP-989: RocksDB Iterator
> Metrics|https://cwiki.apache.org/confluence/display/KAFKA/KIP-989%3A+RocksDB+Iterator+Metrics]
> RocksDB {{Iterators}} must be closed after use, to prevent memory leaks due
> to [blocks being "pinned"
> in-memory|https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#blocks-pinned-by-iterators].
> Pinned blocks can currently be tracked via the per-store
> {{block-cache-pinned-usage}} metric. However, it's common [(and even
> recommended)|https://docs.confluent.io/platform/current/streams/developer-guide/memory-mgmt.html#rocksdb]
> to share the Block Cache among all stores in an application, to enable users
> to globally bound native memory used by RocksDB. This results in the
> {{block-cache-pinned-usage}} reporting the same memory usage for every store
> in the application, irrespective of which store is actually pinning blocks in
> the block cache.
> To aid users in finding leaked Iterators, as well as identifying the cause of
> a high number of pinned blocks, we introduce two new metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
