[ 
https://issues.apache.org/jira/browse/KAFKA-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125016#comment-18125016
 ] 

Andrew Schofield commented on KAFKA-21241:
------------------------------------------

I don't think KIP-1261 has really got off the ground yet. It does seem to me 
that we could accommodate increasing the maximum number of members in a share 
group, in particular now the way that rebalancing in the group coordinator has 
got more sophisticated.

> Increase the number of consumers in the share consumer group
> ------------------------------------------------------------
>
>                 Key: KAFKA-21241
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21241
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Prakhar Kulshrestha
>            Priority: Major
>
> Currently the max  number of consumers in a share consumer group is default 
> to 200 and max to 1000.  In bigger platforms where batch processing along 
> with slow consumer use cases limiting to just 1000 consumer is very 
> restrictive.{*}{*}
> One of the scenario is a batch processing of millions of records and having 
> to parallel process 20-30k records in parallel using spring kafka concurrency 
> feature would mean we would need at least 20-30k consumers in a consumer 
> groups with around 1k pods. Current 1000 limit just let it have ~60pods.
> Request this to be reevaluated and increase the number of consumers that a 
> share group can have.
> *group.share.max.size*



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to