[
https://issues.apache.org/jira/browse/KAFKA-21366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicole Du updated KAFKA-21366:
------------------------------
Description:
Tools that replicate consumer group offsets across clusters, such as
MirrorMaker 2 and the group offset sync in KIP-1279, list groups using
{{Admin.listGroups}} and then fetch their offsets using
{{{}Admin.listConsumerGroupOffsets{}}}. In large deployments, many groups may
have members but no committed offsets, resulting in unnecessary OffsetFetch
processing and additional load on group coordinators.
This issue tracks KIP-1388, which proposes adding a request-side filter to the
ListGroups API (version 6) and a corresponding
{{ListGroupsOptions.requireCommittedOffsets(boolean)}} option. When enabled,
the filter allows the server to return only groups with at least one committed
offset, reducing unnecessary offset fetches.
KIP:
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-1388%3A+Add+a+committed-offsets+filter+to+ListGroups]
was:
Tools that replicate consumer group offsets between clusters (MirrorMaker 2,
the group offset sync in KIP-1279, and others built on the Admin API) list
every group with Admin.listGroups and then fetch offsets per group with
Admin.listConsumerGroupOffsets. In large deployments thousands of groups have
members but no committed offsets, so every OffsetFetch for them is wasted work
on the group coordinators.
This ticket tracks the KIP that adds a request-side filter to the ListGroups
API (version 6) and a matching
ListGroupsOptions.requireCommittedOffsets(boolean) option, so that only groups
with at least one committed offset are returned.
KIP:
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-1388%3A+Add+a+committed-offsets+filter+to+ListGroups]
> KIP-1388: Add a committed-offsets filter to ListGroups
> ------------------------------------------------------
>
> Key: KAFKA-21366
> URL: https://issues.apache.org/jira/browse/KAFKA-21366
> Project: Kafka
> Issue Type: Improvement
> Components: clients, group-coordinator
> Reporter: Nicole Du
> Assignee: Nicole Du
> Priority: Major
> Labels: kip
>
> Tools that replicate consumer group offsets across clusters, such as
> MirrorMaker 2 and the group offset sync in KIP-1279, list groups using
> {{Admin.listGroups}} and then fetch their offsets using
> {{{}Admin.listConsumerGroupOffsets{}}}. In large deployments, many groups may
> have members but no committed offsets, resulting in unnecessary OffsetFetch
> processing and additional load on group coordinators.
> This issue tracks KIP-1388, which proposes adding a request-side filter to
> the ListGroups API (version 6) and a corresponding
> {{ListGroupsOptions.requireCommittedOffsets(boolean)}} option. When enabled,
> the filter allows the server to return only groups with at least one
> committed offset, reducing unnecessary offset fetches.
> KIP:
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-1388%3A+Add+a+committed-offsets+filter+to+ListGroups]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
