Lucy Liu created KAFKA-20688:
--------------------------------
Summary: Memory leak in
RocksDBTimestampedStoreWithHeaders.openRocksDB when probing column families
Key: KAFKA-20688
URL: https://issues.apache.org/jira/browse/KAFKA-20688
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 4.3.0
Reporter: Lucy Liu
Fix For: 4.3.1
`RocksDBTimestampedStoreWithHeaders.openRocksDB(...)` allocates a `new
ColumnFamilyOptions()` solely to construct an `Options` for a
`RocksDB.listColumnFamilies(...)` probe call. The temporary
`ColumnFamilyOptions` instance is not referenced anywhere outside the
try-with-resources block, and `Options.close()` does not cascade-close the
inner `ColumnFamilyOptions` in the RocksDB JNI contract. The instance is
unreachable from any close path, leaking one native `LRUCache` block cache per
`openRocksDB` invocation. Each new windowed-store segment open compounds the
leak.
## Affected code
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStoreWithHeaders.java#L75-L80
```
final List<byte[]> existingCFs;
try (final Options options = new Options(dbOptions, new
ColumnFamilyOptions())) {
existingCFs = RocksDB.listColumnFamilies(options,
dbDir.getAbsolutePath());
} catch (final RocksDBException e) {
throw new ProcessorStateException("Error listing column families for
store " + name, e);
}
```
This path is only reached when header-aware state stores are enabled
(KIP-1271). Workloads that do not enable that feature are unaffected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
