[
https://issues.apache.org/jira/browse/KAFKA-21360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
이선민 updated KAFKA-21360:
------------------------
Description:
SimpleHeterogeneousAssignmentBuilder#build computes the assignment for each
topic in the union of the members' subscriptions. A topic that no member
subscribes to anymore is never visited, so a member whose existing assignment
contains it gets its old partitions for that topic carried over into the new
target assignment.
For the same reason, the {{else}} branch in
{{TopicAssignmentPartialBuilder#revokeUnassignablePartitions}} that removes the
topic is
unreachable, since {{topicId}} always comes from {{subscribedTopicIds}} there.
The homogeneous builder handles this case correctly
{{SimpleHomogeneousAssignmentBuilder}}
iterates over each member's existing assignment and removes topics that are no
longer subscribed.
This behavior has been present since the builder was introduced by KAFKA-19363
(https://github.com/apache/kafka/pull/20074), and is present in 4.2.0.
h3. Reproduction
Invoke {{SimpleAssignor#assign}} with the following group specification:
* Topics: T1 (4 partitions), T2 (2 partitions), T3 (2 partitions)
* Member A subscribes to {T1}, existing assignment {T1=[0], T2=[0]}
* Member B subscribes to {T1, T3}, existing assignment {T1=[1], T2=[1]}
* Subscription type: HETEROGENEOUS
Actual assignor output (one observed run):
{noformat}
A={T2=[0], T1=[0, 2]}
B={T2=[1], T1=[1, 3], T3=[0, 1]}
{noformat}
Expected result: T2 is revoked from both members.
h3. Impact
Since KAFKA-19431, {{ShareGroupAssignmentBuilder}} filters out topics outside
each member's subscription when reconciling the target assignment into the
member's current assignment, so these stale partitions are not passed on to
members.
This filtering does not modify the stored target assignment. The stale entries
can therefore
survive subsequent rebalances while the group remains heterogeneous and no
member subscribes to the topic.
This was found while adding tests in
https://github.com/apache/kafka/pull/23874, where it was confirmed as a bug in
review (https://github.com/apache/kafka/pull/23874#issuecomment-6394163465).
was:
union of the members' subscriptions. A topic that no member subscribes to
anymore is never visited, so a member whose existing assignment contains it
gets its old partitions for that topic carried over into the new target
assignment.
For the same reason, the {{else}} branch in
{{TopicAssignmentPartialBuilder#revokeUnassignablePartitions}} that removes the
topic is
unreachable, since {{topicId}} always comes from {{subscribedTopicIds}} there.
The homogeneous builder handles this case correctly
{{SimpleHomogeneousAssignmentBuilder}}
iterates over each member's existing assignment and removes topics that are no
longer subscribed.
This behavior has been present since the builder was introduced by KAFKA-19363
(https://github.com/apache/kafka/pull/20074), and is present in 4.2.0.
h3. Reproduction
Invoke {{SimpleAssignor#assign}} with the following group specification:
* Topics: T1 (4 partitions), T2 (2 partitions), T3 (2 partitions)
* Member A subscribes to {T1}, existing assignment {T1=[0], T2=[0]}
* Member B subscribes to {T1, T3}, existing assignment {T1=[1], T2=[1]}
* Subscription type: HETEROGENEOUS
Actual assignor output (one observed run):
{noformat}
A={T2=[0], T1=[0, 2]}
B={T2=[1], T1=[1, 3], T3=[0, 1]}
{noformat}
Expected result: T2 is revoked from both members.
h3. Impact
Since KAFKA-19431, {{ShareGroupAssignmentBuilder}} filters out topics outside
each member's subscription when reconciling the target assignment into the
member's current assignment, so these stale partitions are not passed on to
members.
This filtering does not modify the stored target assignment. The stale entries
can therefore
survive subsequent rebalances while the group remains heterogeneous and no
member subscribes to the topic.
This was found while adding tests in
https://github.com/apache/kafka/pull/23874, where it was confirmed as a bug in
review (https://github.com/apache/kafka/pull/23874#issuecomment-6394163465).
> Heterogeneous simple share assignor keeps partitions of topics that no member
> subscribes to
> -------------------------------------------------------------------------------------------
>
> Key: KAFKA-21360
> URL: https://issues.apache.org/jira/browse/KAFKA-21360
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator
> Affects Versions: 4.2.0
> Reporter: 이선민
> Assignee: 이선민
> Priority: Trivial
>
> SimpleHeterogeneousAssignmentBuilder#build computes the assignment for each
> topic in the union of the members' subscriptions. A topic that no member
> subscribes to anymore is never visited, so a member whose existing assignment
> contains it gets its old partitions for that topic carried over into the new
> target assignment.
> For the same reason, the {{else}} branch in
> {{TopicAssignmentPartialBuilder#revokeUnassignablePartitions}} that removes
> the topic is
> unreachable, since {{topicId}} always comes from {{subscribedTopicIds}} there.
> The homogeneous builder handles this case correctly
> {{SimpleHomogeneousAssignmentBuilder}}
> iterates over each member's existing assignment and removes topics that are
> no longer subscribed.
> This behavior has been present since the builder was introduced by KAFKA-19363
> (https://github.com/apache/kafka/pull/20074), and is present in 4.2.0.
> h3. Reproduction
> Invoke {{SimpleAssignor#assign}} with the following group specification:
> * Topics: T1 (4 partitions), T2 (2 partitions), T3 (2 partitions)
> * Member A subscribes to {T1}, existing assignment {T1=[0], T2=[0]}
> * Member B subscribes to {T1, T3}, existing assignment {T1=[1], T2=[1]}
> * Subscription type: HETEROGENEOUS
> Actual assignor output (one observed run):
> {noformat}
> A={T2=[0], T1=[0, 2]}
> B={T2=[1], T1=[1, 3], T3=[0, 1]}
> {noformat}
> Expected result: T2 is revoked from both members.
> h3. Impact
> Since KAFKA-19431, {{ShareGroupAssignmentBuilder}} filters out topics outside
> each member's subscription when reconciling the target assignment into the
> member's current assignment, so these stale partitions are not passed on to
> members.
> This filtering does not modify the stored target assignment. The stale
> entries can therefore
> survive subsequent rebalances while the group remains heterogeneous and no
> member subscribes to the topic.
> This was found while adding tests in
> https://github.com/apache/kafka/pull/23874, where it was confirmed as a bug
> in review
> (https://github.com/apache/kafka/pull/23874#issuecomment-6394163465).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
