Hi all—just chiming in here since we built something like this inside
LittleHorse.
Another alternative option to implement TTL which does not require scanning
through all stores is to "hijack" the `.put()` implementation to insert two
keys:
1. nativeStore.put("values/" + key, value);
2. nativeStore.put("timestamps/" + timestamp, key);
Putting the `timestamp` can be as simple as just printing out the
leading-zero-padded epoch time in milliiseconds.
Every punctuation interval (eg. 10s, 1m, whatever you want—should be
configurable—), you simply do a range scan from ["timestamp/0",
"timestamp/" + System.currentTimeMillis() or record.timestamp() ) and
delete each key that is returned. That avoids scanning the full store, and
in fact bounds the rate of things processed in the punctuation to roughly
the rate at which you are processing them from the input topics.
We do something like this in LittleHorse, actually, and it works quite well
(note that we have a separate rocksdb for the timestamps but that's a
really long story which is irrelevant to this discussion).
As for the actual store API, I'd recommend making the constructor / builder
of the RocksDBTTLStore contain a configurable punctuation interval.
May the Force be with you!
Colt McNealy
Founder and Managing Member
https://littlehorse.io
On 2025/10/07 05:25:32 Ankur Sinha wrote:
> Hello,
>
> I’d like to start the discussion 🙂 for KIP-1225: Add Optional TTL
Support to Kafka Streams State Stores<
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1225%3A+Add+Optional+TTL+Support+to+Kafka+Streams+State+Stores
>
> JIRA: https://issues.apache.org/jira/browse/KAFKA-19759<
https://issues.apache.org/jira/browse/KAFKA-19759>
> KIP Wiki:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1225%3A+Add+Optional+TTL+Support+to+Kafka+Streams+State+Stores
>
> Regards,
> Ankur Sinha
>
> ________________________________
> From: Matthias J. Sax <[email protected]>
> Sent: Tuesday, October 7, 2025 1:04 AM
> To: [email protected] <[email protected]>
> Subject: [EXT] Re: [DISCUSS] KIP-19759: Add built-in TTL (Time-to-Live)
support for Kafka Streams State Stores
>
> Warning External sender Do not click on any links or open any attachments
unless you trust the sender and know the content is safe.
>
> If you want to start a KIP, you need to write one :) A Jira ticket is
> not a KIP
>
>
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
<
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>
>
>
> -Matthias
>
> This email was screened for spam and malicious content but exercise
caution anyway.
>
>
>
> On 10/6/25 10:26 AM, Ankur Sinha wrote:
> > Hi all,
> >
> > I’d like to start the discussion for KAFKA-19759: Add built-in TTL
(Time-to-Live) support for Kafka Streams State Stores.
> >
> > JIRA: https://issues.apache.org/jira/browse/KAFKA-19759<
https://issues.apache.org/jira/browse/KAFKA-19759>
> > KIP Wiki: (will be created shortly)
> >
> > Streams users often need per-key TTL functionality (e.g., for
cache-like state, per-key deduplication, or automatic cleanup).
> > Currently, this requires manual punctuators and tombstones.
> >
> > This KIP proposes adding a built-in `withTtl(Duration ttl)` option to
state stores,
> > which would handle key expiration and changelog consistency
automatically.
> >
> > Please see the JIRA for full details and general analysis.
> > Looking forward for a discussion and talk.
> >
> > Best regards,
> > Ankur Sinha
> >
>
