TaiJuWu created KAFKA-21440:
-------------------------------
Summary: Share consumer keeps fetching from the old leader after
KAFKA_STORAGE_ERROR
Key: KAFKA-21440
URL: https://issues.apache.org/jira/browse/KAFKA-21440
Project: Kafka
Issue Type: Bug
Reporter: TaiJuWu
Assignee: TaiJuWu
When the log dir that holds a partition's leader replica fails, the broker
keeps running and returns KAFKA_STORAGE_ERROR for the partition, and the
leadership moves to another broker. A share consumer that was already fetching
the partition never moves to the new leader. It keeps sending ShareFetch to the
old leader, thousands of times per second, and receives no more records from
the partition. New share consumers, the classic consumer and the new consumer
all move to the new leader.
*Steps to reproduce*
# Start a share consumer on a partition with replicas on brokers that have more
than one log dir.
# Fail the log dir of the leader replica. The broker keeps running and the
leadership moves.
# Produce records.
The share consumer gets no records. It keeps sending ShareFetch to the old
leader, which returns KAFKA_STORAGE_ERROR.
*Cause*
The share consumer sends ShareFetch to the leader cached in
\{{shareSessionLeaderMap}}, not to the leader in the metadata. The cache is
only replaced on NOT_LEADER_OR_FOLLOWER, FENCED_LEADER_EPOCH and
UNKNOWN_TOPIC_*. KAFKA_STORAGE_ERROR updates the metadata but not the cache.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)