[
https://issues.apache.org/jira/browse/KAFKA-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125019#comment-18125019
]
Muralidhar Basani commented on KAFKA-21241:
-------------------------------------------
[~schofielaj] I would be happy to write the KIP, if you're fine with it.
However need your inputs on the coordinator side and benchmarks.
> Increase the number of consumers in the share consumer group
> ------------------------------------------------------------
>
> Key: KAFKA-21241
> URL: https://issues.apache.org/jira/browse/KAFKA-21241
> Project: Kafka
> Issue Type: Improvement
> Reporter: Prakhar Kulshrestha
> Priority: Major
>
> Currently the max number of consumers in a share consumer group is default
> to 200 and max to 1000. In bigger platforms where batch processing along
> with slow consumer use cases limiting to just 1000 consumer is very
> restrictive.{*}{*}
> One of the scenario is a batch processing of millions of records and having
> to parallel process 20-30k records in parallel using spring kafka concurrency
> feature would mean we would need at least 20-30k consumers in a consumer
> groups with around 1k pods. Current 1000 limit just let it have ~60pods.
> Request this to be reevaluated and increase the number of consumers that a
> share group can have.
> *group.share.max.size*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)