merlimat commented on issue #6791: URL: https://github.com/apache/pulsar/pull/6791#issuecomment-618176136
> If users want to more evenly distributed, they will increase the points. I'm worried about too many consumers(e.g. 5000 consumers) may result in too many nodes in the TreeMap. Regarding many consumers, sure the default of 100 points could be overkill, since we'd be fine with 1 point per consumer. It would be easy though to dynamically change the number of points in the ring based on the number of consumers. The only part that we need to be careful during this operation is the same as described in #6554, maintaining the ordering after the hash ranges get readjusted. I'd say to start with current simple approach, and make it dynamic if it becomes a pain point. > I'm not sure if this will require more CPU resources, related to [#6647 (comment)](https://github.com/apache/pulsar/pull/6647#issuecomment-618110187). I will test it later. I have a 2nd PR ready (based on top of this one) to address that and other issues on the dispatcher code. > Or, can we add a configuration in the broker.conf? so that we don't need to introduce more options at the client-side. I met some users who already use Key_Shared subscription in production. I think we'd better give the user a transition time and we can deprecate the current implementation, change the default value in the broker.conf and finally delete the source code. The code for the consumer selection is pretty much self-contained and a pretty basic implementation of consistent hashing. For what is worth, this new implementation is also already running in production without issues. That also doesn't remove the fact that the current implementation can easily lead to very imbalanced traffic distribution. In fact, the very first user we got to use key-shared feature got bit by this. That should be regarded as a bug in any case. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
