[
https://issues.apache.org/jira/browse/KAFKA-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sanghyeok An updated KAFKA-21513:
---------------------------------
Component/s: group-coordinator
streams
> Streams internal topic creation retries can omit topics and prevent group
> startup
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-21513
> URL: https://issues.apache.org/jira/browse/KAFKA-21513
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator, streams
> Reporter: sanghyeok An
> Assignee: sanghyeok An
> Priority: Minor
>
> After Streams internal topic creation fails, subsequent retries can omit the
> affected topics even after the backoff expires. As a result, internal topics
> may remain missing after the original failure condition is resolved, leaving
> the group in {{{}MISSING_INTERNAL_TOPICS{}}}.
> *Scenario*
> # A Streams group starts with its source topics already present, but a
> required internal repartition topic does not exist.
> # The Group Coordinator caches the internal topic creation information in
> {{{}ConfiguredTopology{}}}, and the broker uses it to send a {{CreateTopics}}
> request to the controller.
> # A creation request using the topic objects cached in the existing group
> fails due to a temporary condition. For example, creation can fail when the
> replication factor is 3 but only two brokers are available.
> # Another broker becomes available, allowing topic creation to succeed, and
> the error backoff expires. The topology and relevant topic metadata remain
> unchanged, so the coordinator continues using the existing cache.
> # During a subsequent heartbeat, internal topic creation is retried, but the
> previously attempted topic is omitted from the actual {{CreateTopics}}
> request.
> # The internal topic remains missing, preventing the group from becoming
> ready. Subsequent retries can encounter the same problem while the same cache
> is retained.
> The retry call itself executes, but the controller does not receive a
> creation entry for the affected topic. If it is the only topic being retried,
> the request contains an empty {{Topics}} array. Heartbeats can continue
> succeeding while the client waits for internal topics to become available.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)