TaiJuWu created KAFKA-21517:
-------------------------------
Summary: Share consumer sends ShareFetch in a tight loop on
TOPIC_AUTHORIZATION_FAILED and recovers slowly after the ACL is restored
Key: KAFKA-21517
URL: https://issues.apache.org/jira/browse/KAFKA-21517
Project: Kafka
Issue Type: Bug
Reporter: TaiJuWu
Assignee: TaiJuWu
While a share consumer is not authorized to read a topic, it sends ShareFetch
requests in a tight loop. After the ACL is restored, it keeps throwing
\{{TopicAuthorizationException}} from \{{poll()}} for a
long time before it receives records again.
*Steps*
# A cluster with 3 brokers, SASL_PLAINTEXT and \{{StandardAuthorizer}}, and
the default broker and share group configs, except
\{{share.auto.offset.reset=earliest}}.
# A user with READ on the topic and the group consumes with two share
consumers.
# Remove the topic READ ACL for 10 seconds, then add it back, and produce
more records.
*Expected:* the consumers back off while they are not authorized, and receive
the new records soon after the ACL is restored.
*Actual:*
* While the ACL is removed, each consumer sends about 1,400 ShareFetch
requests per second.
* After the ACL is restored, each consumer throws
\{{TopicAuthorizationException}} from 523 to 806 consecutive \{{poll()}} calls
before it receives records. With a poll every 500 ms, it does not recover
within minutes.
* A new share consumer with the same user and group, and a classic consumer,
read the records right away.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)