[
https://issues.apache.org/jira/browse/KAFKA-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Huang reassigned KAFKA-21397:
---------------------------------
Assignee: Ken Huang
> a single large batch bigger than the quota's allowed burst can stall
> consumers indefinitely
> -------------------------------------------------------------------------------------------
>
> Key: KAFKA-21397
> URL: https://issues.apache.org/jira/browse/KAFKA-21397
> Project: Kafka
> Issue Type: Bug
> Components: core
> Reporter: Jun Rao
> Assignee: Ken Huang
> Priority: Major
>
> How the stall happens: The fetch size is capped to fit the quota. For
> consumer fetches, KafkaApis.scala:740-748 caps fetchMaxBytes at
> maxValueInQuotaWindow. That is quota × (quota.window.num − 1) ×
> quota.window.size.seconds (ClientQuotaManager.java:377-383), so about 10× the
> per-second quota with the defaults.
> The first batch ignores the cap. ReplicaManager.readFromLog reads the first
> non-empty partition with minOneMessage = true
> (ReplicaManager.scala:1896-1906).
> KafkaApis.scala:704-721 records the response size. That alone puts the client
> over its quota, so the throttle time is greater than 0. The broker then
> removes the bytes it just recorded and returns an empty throttled response.
> The consumer retries from the same offset after the throttle time. The quota
> is still configured based on the configured window sizes and doesn't change.
> The consumer gets the same empty response.
> Share fetch has the same pattern (KafkaApis.scala:3559-3561 and :4327).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
