[
https://issues.apache.org/jira/browse/KAFKA-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088648#comment-18088648
]
Nilesh Kumar commented on KAFKA-20688:
--------------------------------------
[~lucliu] I'd like to take this issue and work on a fix. Thanks.
> Memory leak in RocksDBTimestampedStoreWithHeaders.openRocksDB when probing
> column families
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-20688
> URL: https://issues.apache.org/jira/browse/KAFKA-20688
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.3.0
> Reporter: Lucy Liu
> Priority: Blocker
> Fix For: 4.3.1
>
>
> `RocksDBTimestampedStoreWithHeaders.openRocksDB(...)` allocates a `new
> ColumnFamilyOptions()` solely to construct an `Options` for a
> `RocksDB.listColumnFamilies(...)` probe call. The temporary
> `ColumnFamilyOptions` instance is not referenced anywhere outside the
> try-with-resources block, and `Options.close()` does not cascade-close the
> inner `ColumnFamilyOptions` in the RocksDB JNI contract. The instance is
> unreachable from any close path, leaking one native `LRUCache` block cache
> per `openRocksDB` invocation. Each new windowed-store segment open compounds
> the leak.
>
> *Affected code*
> [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStoreWithHeaders.java#L75-L80]
> {code:java}
> final List<byte[]> existingCFs;
> try (final Options options = new Options(dbOptions, new
> ColumnFamilyOptions())) {
> existingCFs = RocksDB.listColumnFamilies(options, dbDir.getAbsolutePath());
> } catch (final RocksDBException e) {
> throw new ProcessorStateException("Error listing column families for store
> " + name, e);
> } {code}
> This path is only reached when header-aware state stores are enabled
> (KIP-1271). Workloads that do not enable that feature are unaffected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
