bbejeck commented on code in PR #22378:
URL: https://github.com/apache/kafka/pull/22378#discussion_r3364968070
##########
streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java:
##########
@@ -311,19 +312,16 @@ private void addValueProvidersToMetricsRecorder() {
}
/**
- * Creates lightweight {@link ColumnFamilyOptions} for the offsets column
family. The offsets CF
- * stores only a small number of key-value pairs (one per changelog
partition), so it does not
- * need the heavyweight options used for the data CF (large write buffers,
bloom filters,
- * aggressive compaction). Sharing the data CF's options causes
unnecessary write amplification
- * and compaction pressure that can contribute to RocksDB write stalls
under heavy restore I/O.
+ * The offsets CF stores only a small number of key-value pairs
+ * (one per changelog partition), so it does not need the heavyweight
+ * options used for the data CF.
+ * Uses the default options for compaction and max write buffer number.
*/
- protected static ColumnFamilyOptions createOffsetsCFOptions() {
- final ColumnFamilyOptions offsetsCFOptions = new ColumnFamilyOptions();
- offsetsCFOptions.setCompressionType(CompressionType.NO_COMPRESSION);
- offsetsCFOptions.setCompactionStyle(CompactionStyle.LEVEL);
- offsetsCFOptions.setWriteBufferSize(1024 * 1024L); // 1MB — sufficient
for offset metadata
- offsetsCFOptions.setMaxWriteBufferNumber(2);
- return offsetsCFOptions;
+ protected ColumnFamilyOptions createOffsetsCFOptions() {
Review Comment:
Agreed. I've updated the code to initialize the `offsetsCfOptions` in
`openDB`, consistent with the other options (`fOptions`, `wOptions`) and
provided an accessor method. Since it's a read-only object it's safe to share
with all instances and makes the clean-up simpler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
