[
https://issues.apache.org/jira/browse/KAFKA-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743671#comment-17743671
]
Divij Vaidya commented on KAFKA-14912:
--------------------------------------
> do we know size of each individual segment entry in the cache
No. The cache contains indexes which rotate with segment. If the segment has
time based rotation, then the index might be either small or large. Hence, we
don't really know the size.
>From a user perspective, setting this based on size in Bytes makes more sense
>since they can configure it based on allocation of heap size instead of
>relying on max size of a index file.
Does the cache we use (caffeine) support a max size by bytes (instead of max
size by entry)? Please explore
[https://www.javadoc.io/static/com.github.ben-manes.caffeine/caffeine/2.2.0/com/github/benmanes/caffeine/cache/Caffeine.html#maximumWeight]
If yes, we can change our implementation to adhere to a max size in bytes
instead of max size in entries. Then we can use the existing configuration as
well.
> Introduce a configuration for remote index cache size, preferably a dynamic
> config.
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-14912
> URL: https://issues.apache.org/jira/browse/KAFKA-14912
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Reporter: Satish Duggana
> Assignee: Manyanda Chitimbo
> Priority: Major
>
> Context: We need to make the 1024 value here [1] as dynamically configurable
> [1]
> https://github.com/apache/kafka/blob/8d24716f27b307da79a819487aefb8dec79b4ca8/storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java#L119
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
