Eric Chang created KAFKA-21589:
----------------------------------
Summary: Consumer fetch metrics are recorded more than once when a
fetched partition fails to initialize
Key: KAFKA-21589
URL: https://issues.apache.org/jira/browse/KAFKA-21589
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 4.4.0, 4.3.0, 4.2.0, 4.1.0, 4.0.0
Reporter: Eric Chang
Assignee: Eric Chang
{{FetchCollector.initialize()}} reports a partition to
{{FetchMetricsAggregator}} in a {{finally}} block, also when it throws. When
{{initialize()}} throws, the {{CompletedFetch}} can stay in the {{FetchBuffer}}:
* If the same {{collectFetch()}} call already returned records for another
partition, the exception is not thrown and the {{CompletedFetch}} stays in the
buffer.
* If the {{CompletedFetch}} still has bytes, it is not removed even when the
exception is thrown.
The {{CompletedFetch}} is then reported again by the next {{poll()}}, which
calls {{initialize()}} again, or by {{drain()}} when the partition is no longer
assigned ({{FetchBuffer.retainAll}}).
{{FetchMetricsAggregator}} records the metrics of the whole fetch response when
every partition of the response has reported. After that, each extra report
records them again. So, at the client level and the topic level:
* {{records-consumed-total}}, {{records-consumed-rate}},
{{bytes-consumed-total}} and {{bytes-consumed-rate}} are too high.
* {{fetch-size-avg}} and {{records-per-request-avg}} count that response more
than once.
Example: one fetch response has 10 records for partition A and
{{TOPIC_AUTHORIZATION_FAILED}} for partition B. The first {{poll()}} returns
the 10 records of A and keeps B in the buffer. The second {{poll()}} throws
{{TopicAuthorizationException}}. After the second {{poll()}},
{{records-consumed-total}} is 20 instead of 10.
Other errors that can do the same: {{OFFSET_OUT_OF_RANGE}} without an offset
reset policy, and a non-empty partition with no complete record batch.
The same behavior exists in older releases, in {{Fetcher.java}} (checked back
to 2.0.0).
Proposed fix: {{FetchMetricsAggregator.record()}} ignores a partition that has
already reported.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)