Kamal Chandraprakash created KAFKA-15777:
--------------------------------------------
Summary: Configurable remote fetch bytes per partition from
Consumer
Key: KAFKA-15777
URL: https://issues.apache.org/jira/browse/KAFKA-15777
Project: Kafka
Issue Type: Task
Reporter: Kamal Chandraprakash
Assignee: Kamal Chandraprakash
A consumer can configure the amount of local bytes to read from each partition
in the FETCH request.
{{max.partition.fetch.bytes}} = 1 MB
Similar to this, the consumer should be able to configure
{{max.remote.partition.fetch.bytes}} = 4 MB.
While handling the {{FETCH}} request, if we encounter a partition to read data
from remote storage, then rest of the partitions in the request are ignored.
Essentially, we are serving only 1 MB of remote data per FETCH request when all
the partitions in the request are to be served from the remote storage.
Providing one more configuration to the client help the user to tune the values
depends on their storage plugin. The user might want to optimise the number of
calls to remote storage vs amount of bytes returned back to the client in the
FETCH response.
[https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/ReplicaManager.scala#L1454]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
