[
https://issues.apache.org/jira/browse/KAFKA-20187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirk True updated KAFKA-20187:
------------------------------
Description:
When {{AsyncKafkaConsumer.currentLag()}} is called,
{{SubscriptionState.requestPartitionEndOffset()}} is called, which sets a
per-partition flag named {{endOffsetRequested}}. This flag is later cleared by
{{SubscriptionState.updateLastStableOffset()}}/{{SubscriptionState.updateHighWatermark()}}
when a successful {{LIST_OFFSETS}} call is received. However, if the partition
has an error code in the response, or the {{LIST_OFFSETS}} call itself fails,
neither update method is called, and {{endOffsetRequested}} remains set. The
problem is, on _any subsequent_ call to {{AsyncKafkaConsumer.currentLag()}},
the logic will see that {{endOffsetRequested}} is already set, and the offset
request will be silently ignored.
The fix is to ensure that {{endOffsetRequested}} is cleared out even on failure
cases.
Note: this is the companion ticket to KAFKA-20131 which affects the
{{ClassicKafkaConsumer}}.
was:
When {{ClassicKafkaConsumer.currentLag()}} is called,
{{SubscriptionState.requestPartitionEndOffset()}} is called, which sets a
per-partition flag named {{endOffsetRequested}}. This flag is later cleared by
{{SubscriptionState.updateLastStableOffset()}}/{{SubscriptionState.updateHighWatermark()}}
when a successful {{LIST_OFFSETS}} call is received. However, if the partition
has an error code in the response, or the {{LIST_OFFSETS}} call itself fails,
neither update method is called, and {{endOffsetRequested}} remains set. The
problem is, on _any subsequent_ call to {{ClassicKafkaConsumer.currentLag()}},
the logic will see that {{endOffsetRequested}} is already set, and the offset
request will be silently ignored.
The fix is to ensure that {{endOffsetRequested}} is cleared out even on failure
cases.
> AsyncKafkaConsumer does not clear endOffsetRequested flag on failed
> LIST_OFFSETS calls
> --------------------------------------------------------------------------------------
>
> Key: KAFKA-20187
> URL: https://issues.apache.org/jira/browse/KAFKA-20187
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.0.0, 3.9.1, 4.2.0
> Reporter: Kirk True
> Assignee: Kirk True
> Priority: Critical
> Fix For: 4.3.0
>
>
> When {{AsyncKafkaConsumer.currentLag()}} is called,
> {{SubscriptionState.requestPartitionEndOffset()}} is called, which sets a
> per-partition flag named {{endOffsetRequested}}. This flag is later cleared
> by
> {{SubscriptionState.updateLastStableOffset()}}/{{SubscriptionState.updateHighWatermark()}}
> when a successful {{LIST_OFFSETS}} call is received. However, if the
> partition has an error code in the response, or the {{LIST_OFFSETS}} call
> itself fails, neither update method is called, and {{endOffsetRequested}}
> remains set. The problem is, on _any subsequent_ call to
> {{AsyncKafkaConsumer.currentLag()}}, the logic will see that
> {{endOffsetRequested}} is already set, and the offset request will be
> silently ignored.
> The fix is to ensure that {{endOffsetRequested}} is cleared out even on
> failure cases.
> Note: this is the companion ticket to KAFKA-20131 which affects the
> {{ClassicKafkaConsumer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
