TaiJuWu created KAFKA-21440:
-------------------------------

             Summary: Share consumer keeps fetching from the old leader after 
KAFKA_STORAGE_ERROR
                 Key: KAFKA-21440
                 URL: https://issues.apache.org/jira/browse/KAFKA-21440
             Project: Kafka
          Issue Type: Bug
            Reporter: TaiJuWu
            Assignee: TaiJuWu


When the log dir that holds a partition's leader replica fails, the broker 
keeps running and returns KAFKA_STORAGE_ERROR for the partition, and the 
leadership moves to another broker. A share consumer that was already fetching 
the partition never moves to the new leader. It keeps sending ShareFetch to the 
old leader, thousands of times per second, and receives no more records from 
the partition. New share consumers, the classic consumer and the new consumer 
all move to the new leader.

*Steps to reproduce*

# Start a share consumer on a partition with replicas on brokers that have more 
than one log dir.
# Fail the log dir of the leader replica. The broker keeps running and the 
leadership moves.
# Produce records.

The share consumer gets no records. It keeps sending ShareFetch to the old 
leader, which returns KAFKA_STORAGE_ERROR.

*Cause*

The share consumer sends ShareFetch to the leader cached in 
\{{shareSessionLeaderMap}}, not to the leader in the metadata. The cache is 
only replaced on NOT_LEADER_OR_FOLLOWER, FENCED_LEADER_EPOCH and 
UNKNOWN_TOPIC_*. KAFKA_STORAGE_ERROR updates the metadata but not the cache.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to