[
https://issues.apache.org/jira/browse/KAFKA-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Bejeck resolved KAFKA-20688.
---------------------------------
Resolution: Fixed
> Memory leak in RocksDBTimestampedStoreWithHeaders.openRocksDB when probing
> column families
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-20688
> URL: https://issues.apache.org/jira/browse/KAFKA-20688
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.3.0
> Reporter: Lucy Liu
> Assignee: Matthias J. Sax
> Priority: Blocker
> Fix For: 4.3.1
>
>
> `RocksDBTimestampedStoreWithHeaders.openRocksDB(...)` allocates a `new
> ColumnFamilyOptions()` solely to construct an `Options` for a
> `RocksDB.listColumnFamilies(...)` probe call. The temporary
> `ColumnFamilyOptions` instance is not referenced anywhere outside the
> try-with-resources block, and `Options.close()` does not cascade-close the
> inner `ColumnFamilyOptions` in the RocksDB JNI contract. The instance is
> unreachable from any close path, leaking one native `LRUCache` block cache
> per `openRocksDB` invocation. Each new windowed-store segment open compounds
> the leak.
>
> *Affected code*
> [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStoreWithHeaders.java#L75-L80]
> {code:java}
> final List<byte[]> existingCFs;
> try (final Options options = new Options(dbOptions, new
> ColumnFamilyOptions())) {
> existingCFs = RocksDB.listColumnFamilies(options, dbDir.getAbsolutePath());
> } catch (final RocksDBException e) {
> throw new ProcessorStateException("Error listing column families for store
> " + name, e);
> } {code}
> This path is only reached when header-aware state stores are enabled
> (KIP-1271). Workloads that do not enable that feature are unaffected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
