[ 
https://issues.apache.org/jira/browse/KAFKA-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885498#comment-16885498
 ] 

James Ritt commented on KAFKA-4212:
-----------------------------------

Thanks [~ableegoldman] & [~mjsax], you both make good points. Unfortunately the 
`Map` argument to `setConfig()` appears to be populated from 
`org.apache.kafka.streams.processor.ProcessorContext#appConfigs`, meaning it's 
effectively just pulling in the current `StreamsConfig`.

In order to keep the configs consolidated as per Sophie's request, what do we 
think about defining a new method on the `RocksDBConfigSetter` interface?

We could possibly create it with a default (for backwards compatibility): 
something like `default Integer getTtl(final String storeName, final Options 
options) \{return null;}`, with the idea that we'd check this method's result 
to determine which RocksDB constructor to call.

> Add a key-value store that is a TTL persistent cache
> ----------------------------------------------------
>
>                 Key: KAFKA-4212
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4212
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions: 0.10.0.1
>            Reporter: Elias Levy
>            Priority: Major
>              Labels: api
>
> Some jobs needs to maintain as state a large set of key-values for some 
> period of time.  I.e. they need to maintain a TTL cache of values potentially 
> larger than memory. 
> Currently Kafka Streams provides non-windowed and windowed key-value stores.  
> Neither is an exact fit to this use case.  
> The {{RocksDBStore}}, a {{KeyValueStore}}, stores one value per key as 
> required, but does not support expiration.  The TTL option of RocksDB is 
> explicitly not used.
> The {{RocksDBWindowsStore}}, a {{WindowsStore}}, can expire items via segment 
> dropping, but it stores multiple items per key, based on their timestamp.  
> But this store can be repurposed as a cache by fetching the items in reverse 
> chronological order and returning the first item found.
> KAFKA-2594 introduced a fixed-capacity in-memory LRU caching store, but here 
> we desire a variable-capacity memory-overflowing TTL caching store.
> Although {{RocksDBWindowsStore}} can be repurposed as a cache, it would be 
> useful to have an official and proper TTL cache API and implementation.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Reply via email to