[
https://issues.apache.org/jira/browse/KAFKA-13524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Roesler reassigned KAFKA-13524:
------------------------------------
Assignee: Vicky Papavasileiou
> IQv2: Implement KeyQuery from the RecordCache
> ---------------------------------------------
>
> Key: KAFKA-13524
> URL: https://issues.apache.org/jira/browse/KAFKA-13524
> Project: Kafka
> Issue Type: Sub-task
> Reporter: John Roesler
> Assignee: Vicky Papavasileiou
> Priority: Major
>
> The Record Cache in Kafka Streams is more properly termed a write buffer,
> since it only caches writes, not reads, and its intent is to buffer the
> writes before flushing them in bulk into lower store layers.
> Unlike scan-type queries, which require scanning both the record cache and
> the underlying store and collating the results, the KeyQuery (and any other
> point lookup) can straightforwardly be served from the record cache if it is
> buffered or fall through to the underlying store if not.
> In contrast to scan-type operations, benchmarks reveal that key-based cache
> reads are faster than always skipping the cache as well.
> Therefore, it makes sense to implement a handler in the CachingKeyValueStore
> for the KeyQuery specifically in order to serve fresher key-based lookups.
> Scan queries may also be useful, but their less flattering performance
> profile makes it reasonable to leave them for follow-on work.
> We could add an option to disable cache reads on the KeyQuery, but since they
> seem to be always better, I'm leaning toward just unilaterally serving cached
> records if they exist.
>
> I did a quick POC of this:
> [https://github.com/vvcephei/kafka/pull/new/iqv2-poc-cache-queries]
>
> The internal code of the caching stores should be refactored to share logic
> with the regular store methods. Scan queries will be more complicated, since
> they require merging the cache with the wrapped result.
> There is a bug related to that non-timestamped-store-serde hack (see the
> failing test when you run IQv2StoreIntegrationTest). Even though the inner
> store is not timestamped, the cache returns a timestamped value. We'll have
> to discuss options to fix it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
