Sophie Blee-Goldman created KAFKA-9987:
------------------------------------------
Summary: Add new cooperative assignor optimized for
constant-subscription. group
Key: KAFKA-9987
URL: https://issues.apache.org/jira/browse/KAFKA-9987
Project: Kafka
Issue Type: Improvement
Components: clients
Reporter: Sophie Blee-Goldman
Assignee: Sophie Blee-Goldman
In KIP-429 we added the new CooperativeStickyAssignor which leverages on the
underlying sticky assignment algorithm of the existing StickyAssignor (moved to
AbstractStickyAssignor).
The algorithm is fairly complex as it tries to optimize stickiness while
satisfying perfect balance _in the case individual consumers may be subscribed
to a random subset of the topics._ While it does a pretty good job at what it
promises to do, it doesn't scale well with large numbers of consumers and
partitions.
If we can make the assumption that all consumers are subscribed to the same set
of topics, we can simplify the algorithm greatly and do a sticky-but-balanced
assignment in a single pass. It would be nice to have an additional cooperative
assignor OOTB that performs efficiently for users who know their group will
satisfy this constraint.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
