Hi Matthias, MJS1: When trying to figure out how to record this metric, I figured the simplest and most straightforward way is to update the metric on every put and delete, which would have a nonzero performance impact, thus I decided to add it only at the DEBUG level. I'm not sure how RocksDB collects its equivalent metric, but my understanding is that Kafka Streams just passes up the value it gets from RocksDB and doesn't do any of the collecting itself, which is why I believe the in-memory implementation would be more expensive
MJS2: Good point, I've updated the KIP to reflect this. To repeat what is in the KIP, this metric will be implemented for all classes starting with `Metered` under `streams/state/internals`. For completeness, the metered iterators are able to remove elements, thus they are included here. Thanks, Evan On Mon, Jan 12, 2026 at 6:13 PM Matthias J. Sax <[email protected]> wrote: > Thanks for the KIP Evan. > > MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB > metric is added at INFO level. The KIP says: > > > The addition of this metric will have no effect on existing users. The > performance impact of adding this metric is non-trivial, thus we only > record it at the DEBUG level. > > Why is it expensive? Not clear to me atm. And why would it be cheaper > for the RocksDB case? > > > > > About Lucas question: Not sure if we would need to do anything about it? > Given that we have specific RocksDB metrics, which are not available for > in-memory stores, why should we not have specific in-memory store > metrics? RocksDB metrics are also not containing "rocksdb" in their names? > > As long as it's properly document as "in-memory metric" (and the > description already mentions it explicitly) it might just be fine? > > > > MJS2: The KIP says, we would implement this only for > `MeteredKeyValueStore`. Why not also add it to the other store types > like windowed and others? > > > -Matthias > > > On 12/12/25 4:14 PM, Evan Zhou via dev wrote: > > Hi Lucas, > > > > Thanks for the feedback. > > > > My original intention was to use "estimate" in the metric name as the > > differentiator, but I agree that this is not very clear. How does > changing > > the metric name to "in-memory-state-num-keys" sound? > > > > Thanks, > > Evan > > > > On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev < > [email protected]> > > wrote: > > > >> Hi Evan, > >> > >> thanks for the KIP! > >> > >> If this is specific to in-memory stores, I wonder if we should add > >> this to the metric name? People could become confused that rocksDB > >> state is not showing up. > >> > >> Cheers, > >> Lucas > >> > >> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <[email protected] > > > >> wrote: > >>> > >>> Hi all, > >>> > >>> I'd like to start the discussion for KIP-1250, which adds a new metric > to > >>> track the size of in-memory state stores. Today, a similar metric > exists > >> in > >>> Kafka, but only for RocksDB, and this KIP intends to close that gap. > >>> > >>> https://cwiki.apache.org/confluence/x/noTMFw > >>> > >>> Thanks, > >>> Evan > >> > > > >
