[
https://issues.apache.org/jira/browse/KAFKA-21302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129533#comment-18129533
]
Lucas Brutschy commented on KAFKA-21302:
----------------------------------------
Good point. So the bug triggers:
* {{group.protocol=streams}} and a join, so the topology has a copartition
group.
* Those topics are subscribed by one source node through a collection with
per-JVM iteration order (e.g. {{{}builder.stream(Set.of(...)){}}}), or members
run equivalent but differently written topologies.
* Two topics in that copartition group land in the same {{HashSet}} bucket
(about 6% chance for a pair of topics).
That is more narrow than I originally thought, but it should definitely be
fixed.
> Streams group joins rejected due to copartition topic ordering differences
> --------------------------------------------------------------------------
>
> Key: KAFKA-21302
> URL: https://issues.apache.org/jira/browse/KAFKA-21302
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: sanghyeok An
> Assignee: sanghyeok An
> Priority: Minor
>
> With {{{}group.protocol=streams{}}}, a new application instance can fail to
> join its group even when it uses the same logical topology, depending on the
> order in which source topics are supplied.
> For example, two instances with the same {{application.id}} construct their
> topologies as follows:
> * Instance A joins {{stream(List.of("Aa", "BB"))}} with a stream from topic
> {{{}z{}}}.
> * Instance B joins {{stream(List.of("BB", "Aa"))}} with a stream from topic
> {{{}z{}}}.
> All three topics have the same partition count. Node names, join settings,
> and all other configuration are identical. Only the source topic order
> differs, so both topologies have the same copartition constraints.
> The client sorts the source topic list in the heartbeat, producing {{[Aa, BB,
> z]}} for both instances. However, copartition indexes are collected in Set
> iteration order, so A can send {{[0, 1, 2]}} while B sends {{{}[1, 0, 2]{}}}.
> After A joins and its topology is stored, the coordinator treats B’s
> different index ordering as a topology change and returns:
>
> {{INVALID_REQUEST: Topology updates are not supported yet.}}
> B’s client treats this error as fatal, transitions from {{JOINING}} to
> {{{}FATAL{}}}, and stops sending heartbeats. As a result, adding or replacing
> an instance can fail despite the instances using logically identical
> topologies.
> This occurs when the copartition list ordering actually differs. Reversing an
> arbitrary source topic list does not necessarily trigger the issue. In this
> example, {{Aa}} and {{BB}} have the same Java string hash code, allowing
> their input order to affect Set iteration order.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
