squah-confluent commented on code in PR #23883:
URL: https://github.com/apache/kafka/pull/23883#discussion_r4339751680
##########
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/modern/GroupSpecImpl.java:
##########
@@ -96,11 +104,32 @@ public SubscriptionType subscriptionType() {
*/
@Override
public boolean isPartitionAssigned(Uuid topicId, int partitionId) {
- Map<Integer, String> partitionMap =
invertedMemberAssignment.get(topicId);
- if (partitionMap == null) {
- return false;
+ if (assignedTopicPartitions == null) {
+ assignedTopicPartitions = computeAssignedTopicPartitions();
+ }
+ BitSet assignedPartitions = assignedTopicPartitions.get(topicId);
+ return assignedPartitions != null &&
assignedPartitions.get(partitionId);
+ }
+
+ private Map<Uuid, BitSet> computeAssignedTopicPartitions() {
+ Map<Uuid, BitSet> assignedTopicPartitions = new HashMap<>();
+ for (MemberSubscriptionAndAssignmentImpl member : members.values()) {
+ for (Map.Entry<Uuid, Set<Integer>> entry :
member.partitions().entrySet()) {
+ BitSet partitions =
assignedTopicPartitions.computeIfAbsent(entry.getKey(), topicId -> new BitSet(
+
metadataImage.topicMetadata(topicId).map(CoordinatorMetadataImage.TopicMetadata::partitionCount).orElse(0)
+ ));
+ for (int partition : entry.getValue()) {
+ if (partition >= partitions.size()) {
Review Comment:
We can be more precise in dropping non-existent partitions by doing a 2nd
pass through the map afterwards. It keeps the number of metadata image lookups
limited too.
The homogeneous runs become a tiny bit slower but still an overall
improvement.
`uniform` assignor, `partitionsToMemberRatio=50`, `topicCount=100`, 2 warmup
and 2 measurement iterations of 5s, ms/op.
| memberCount | subscriptionType | TimelineHashMap `build` | Lazy
assignedPartitions `build` | Change |
| ----------- | ---------------- | ----------------------- |
------------------------------- | ------ |
| 5000 | HOMOGENEOUS | 6.180 | 3.886
| -37% |
| 5000 | HETEROGENEOUS | 10.618 | 12.212
| +15% |
| 10000 | HOMOGENEOUS | 14.781 | 8.800
| -40% |
| 10000 | HETEROGENEOUS | 27.686 | 30.261
| +9% |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
