Bill Bejeck created KAFKA-21338:
-----------------------------------
Summary: Use exponential backoff with jitter for internal topic
retries in InternalTopicManager
Key: KAFKA-21338
URL: https://issues.apache.org/jira/browse/KAFKA-21338
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: Bill Bejeck
Fix For: 4.5.0
In Kafka Streams the InternalTopicManager retries internal topic
describe/create/validate on a fixed \{{retry.backoff.ms}} (default 100 ms) until
{\{max.poll.interval.ms}} / 2 (default 150 s). The backoff never grows and has
no jitter. During a
broker outage that is up to ~1,500 retry rounds per application, and
applications on the same
cluster that rebalance after the same outage retry in lockstep.
The producer, consumer and admin clients already back off exponentially with
jitter (KIP-580,
{\{org.apache.kafka.common.utils.ExponentialBackoff}}). InternalTopicManager
should do the same.
Out of scope: the streams rebalance protocol (KIP-1071), where the broker
creates internal topics.
Depends on KAFKA-21033 (https://github.com/apache/kafka/pull/23363), which
moves the backoff back
to one sleep per round in maybeSleep and switches it to time.sleep. This change
should build on
that.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
