[ 
https://issues.apache.org/jira/browse/KAFKA-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125019#comment-18125019
 ] 

Muralidhar Basani commented on KAFKA-21241:
-------------------------------------------

[~schofielaj] I would be happy to write the KIP, if you're fine with it. 
However need your inputs on the coordinator side and benchmarks.

> Increase the number of consumers in the share consumer group
> ------------------------------------------------------------
>
>                 Key: KAFKA-21241
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21241
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Prakhar Kulshrestha
>            Priority: Major
>
> Currently the max  number of consumers in a share consumer group is default 
> to 200 and max to 1000.  In bigger platforms where batch processing along 
> with slow consumer use cases limiting to just 1000 consumer is very 
> restrictive.{*}{*}
> One of the scenario is a batch processing of millions of records and having 
> to parallel process 20-30k records in parallel using spring kafka concurrency 
> feature would mean we would need at least 20-30k consumers in a consumer 
> groups with around 1k pods. Current 1000 limit just let it have ~60pods.
> Request this to be reevaluated and increase the number of consumers that a 
> share group can have.
> *group.share.max.size*



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to