sanghyeok An created KAFKA-21513:
------------------------------------

             Summary: Streams internal topic creation retries can omit topics 
and prevent group startup
                 Key: KAFKA-21513
                 URL: https://issues.apache.org/jira/browse/KAFKA-21513
             Project: Kafka
          Issue Type: Bug
            Reporter: sanghyeok An
            Assignee: sanghyeok An


After Streams internal topic creation fails, subsequent retries can omit the 
affected topics even after the backoff expires. As a result, internal topics 
may remain missing after the original failure condition is resolved, leaving 
the group in {{{}MISSING_INTERNAL_TOPICS{}}}.

*Scenario*
 # A Streams group starts with its source topics already present, but a 
required internal repartition topic does not exist.
 # The Group Coordinator caches the internal topic creation information in 
{{{}ConfiguredTopology{}}}, and the broker uses it to send a {{CreateTopics}} 
request to the controller.
 # A creation request using the topic objects cached in the existing group 
fails due to a temporary condition. For example, creation can fail when the 
replication factor is 3 but only two brokers are available.
 # Another broker becomes available, allowing topic creation to succeed, and 
the error backoff expires. The topology and relevant topic metadata remain 
unchanged, so the coordinator continues using the existing cache.
 # During a subsequent heartbeat, internal topic creation is retried, but the 
previously attempted topic is omitted from the actual {{CreateTopics}} request.
 # The internal topic remains missing, preventing the group from becoming 
ready. Subsequent retries can encounter the same problem while the same cache 
is retained.

The retry call itself executes, but the controller does not receive a creation 
entry for the affected topic. If it is the only topic being retried, the 
request contains an empty {{Topics}} array. Heartbeats can continue succeeding 
while the client waits for internal topics to become available.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to