[ 
https://issues.apache.org/jira/browse/KAFKA-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sanghyeok An updated KAFKA-21513:
---------------------------------
    Component/s: group-coordinator
                 streams

> Streams internal topic creation retries can omit topics and prevent group 
> startup
> ---------------------------------------------------------------------------------
>
>                 Key: KAFKA-21513
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21513
>             Project: Kafka
>          Issue Type: Bug
>          Components: group-coordinator, streams
>            Reporter: sanghyeok An
>            Assignee: sanghyeok An
>            Priority: Minor
>
> After Streams internal topic creation fails, subsequent retries can omit the 
> affected topics even after the backoff expires. As a result, internal topics 
> may remain missing after the original failure condition is resolved, leaving 
> the group in {{{}MISSING_INTERNAL_TOPICS{}}}.
> *Scenario*
>  # A Streams group starts with its source topics already present, but a 
> required internal repartition topic does not exist.
>  # The Group Coordinator caches the internal topic creation information in 
> {{{}ConfiguredTopology{}}}, and the broker uses it to send a {{CreateTopics}} 
> request to the controller.
>  # A creation request using the topic objects cached in the existing group 
> fails due to a temporary condition. For example, creation can fail when the 
> replication factor is 3 but only two brokers are available.
>  # Another broker becomes available, allowing topic creation to succeed, and 
> the error backoff expires. The topology and relevant topic metadata remain 
> unchanged, so the coordinator continues using the existing cache.
>  # During a subsequent heartbeat, internal topic creation is retried, but the 
> previously attempted topic is omitted from the actual {{CreateTopics}} 
> request.
>  # The internal topic remains missing, preventing the group from becoming 
> ready. Subsequent retries can encounter the same problem while the same cache 
> is retained.
> The retry call itself executes, but the controller does not receive a 
> creation entry for the affected topic. If it is the only topic being retried, 
> the request contains an empty {{Topics}} array. Heartbeats can continue 
> succeeding while the client waits for internal topics to become available.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to