[
https://issues.apache.org/jira/browse/KAFKA-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539129#comment-16539129
]
ASF GitHub Bot commented on KAFKA-7146:
---------------------------------------
shunge opened a new pull request #5353: KAFKA-7146: Grouping consumer requests
per consumer coordinator in admin client in describeConsumerGroups
URL: https://github.com/apache/kafka/pull/5353
This is a subtask from KAFKA-6788
(https://issues.apache.org/jira/browse/KAFKA-6788)
Basically the idea is to use thread-safe collections like
ConcurrentLinkedQueue and ConcurrentHashMap to save to result of each
"findCoordinator" response. In the map, we will have the coordinator node Id as
the key, and the value will be the collection of all the belonging groupIds.
When all the groupId processed (either successful or completed
exceptionally), then we will iterate through the map and group the requests by
coordinators.
This PR is mainly for discussion, there are a couple concerns:
1. Code logic: Is there any thing wrong with the logic of this solution?
2. Concurrency issues: can these collections guarantee the thread-safety?
3. Memory issue: using too many collections too store information, should
cut down the memory usage.
4. Efficiency: is this more efficient than the implementation before?
5. Naming issues (I already noticed a typo).
6. More unit tests.
I am open to all kind of suggestions and feedbacks, and will make several
other commits after I receive opinion.
P.S. Is a KIP needed?
@guozhangwang @cmccabe Thank you!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Grouping consumer requests per consumer coordinator in admin client in
> describeConsumerGroups
> ---------------------------------------------------------------------------------------------
>
> Key: KAFKA-7146
> URL: https://issues.apache.org/jira/browse/KAFKA-7146
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Yishun Guan
> Assignee: Yishun Guan
> Priority: Critical
>
> Subtask of KAFKA-6788. Group consumer requests for describeConsumerGroups().
>
> According to KAFKA-6788:
>
> "In KafkaAdminClient, for some requests like describeGroup and deleteGroup,
> we will first try to get the coordinator for each requested group id, and
> then send the corresponding request for that group id. However, different
> group ids could be hosted on the same coordinator, and these requests do
> support multi group ids be sent within the same request. So we can consider
> optimize it by grouping the requests per coordinator destination."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
