[
https://issues.apache.org/jira/browse/KAFKA-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125016#comment-18125016
]
Andrew Schofield commented on KAFKA-21241:
------------------------------------------
I don't think KIP-1261 has really got off the ground yet. It does seem to me
that we could accommodate increasing the maximum number of members in a share
group, in particular now the way that rebalancing in the group coordinator has
got more sophisticated.
> Increase the number of consumers in the share consumer group
> ------------------------------------------------------------
>
> Key: KAFKA-21241
> URL: https://issues.apache.org/jira/browse/KAFKA-21241
> Project: Kafka
> Issue Type: Improvement
> Reporter: Prakhar Kulshrestha
> Priority: Major
>
> Currently the max number of consumers in a share consumer group is default
> to 200 and max to 1000. In bigger platforms where batch processing along
> with slow consumer use cases limiting to just 1000 consumer is very
> restrictive.{*}{*}
> One of the scenario is a batch processing of millions of records and having
> to parallel process 20-30k records in parallel using spring kafka concurrency
> feature would mean we would need at least 20-30k consumers in a consumer
> groups with around 1k pods. Current 1000 limit just let it have ~60pods.
> Request this to be reevaluated and increase the number of consumers that a
> share group can have.
> *group.share.max.size*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)