Divij Vaidya created KAFKA-15084:
------------------------------------
Summary: Remove lock contention in RemoteIndexCache
Key: KAFKA-15084
URL: https://issues.apache.org/jira/browse/KAFKA-15084
Project: Kafka
Issue Type: Sub-task
Components: core
Affects Versions: 3.6.0
Reporter: Divij Vaidya
Assignee: Divij Vaidya
Fix For: 3.6.0
RemoteIndexCache cache is accessed from multiple threads concurrently in the
fetch from consumer code path [1].
Currently, the RemoteIndexCache uses LinkedHashMap as the cache implementation
internally. Since LinkedHashMap is not a thread safe data structure, we use
coarse grained lock on the entire map/cache when writing to the cache.
This means that if a thread if fetching information from a particular segment
from RemoteStorageManager, other threads who are trying to access a different
segment from the cache will also wait for the former thread to complete. This
is due to the usage of global lock in the cache.
This lock contentions leads to decrease in throughput for fetch from consumer
for cases where RSM network call may take more time.
As a goal for this JIRA, we would like to ensure that the threads reading
existing values in the cache do not get blocked when thread updating the cache
is fetching data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
