[
https://issues.apache.org/jira/browse/KAFKA-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
이선민 updated KAFKA-21241:
------------------------
Attachment: assignor-gc-G1.json
assignor-gc-Parallel.json
assignor-sweep-pretouch.json
hashmap-isolated.json
MapBench.java
notes.txt
working-set.json
WorkingSetBench.java
> Increase the number of consumers in the share consumer group
> ------------------------------------------------------------
>
> Key: KAFKA-21241
> URL: https://issues.apache.org/jira/browse/KAFKA-21241
> Project: Kafka
> Issue Type: Improvement
> Reporter: Prakhar Kulshrestha
> Priority: Major
> Attachments: MapBench.java, WorkingSetBench.java,
> assignor-gc-G1.json, assignor-gc-Parallel.json, assignor-sweep-pretouch.json,
> hashmap-isolated.json, notes.txt, working-set.json
>
>
> Currently the max number of consumers in a share consumer group is default
> to 200 and max to 1000. In bigger platforms where batch processing along
> with slow consumer use cases limiting to just 1000 consumer is very
> restrictive.{*}{*}
> One of the scenario is a batch processing of millions of records and having
> to parallel process 20-30k records in parallel using spring kafka concurrency
> feature would mean we would need at least 20-30k consumers in a consumer
> groups with around 1k pods. Current 1000 limit just let it have ~60pods.
> Request this to be reevaluated and increase the number of consumers that a
> share group can have.
> *group.share.max.size*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
