Hi kafka experts, My understanding of static membership is that assuming kubernetes, for example, can provide a fixed number of healthy pods almost always, so that kafka doesn't have to do any rebalancing.
It leads me to think, if the starting point is partition assignment being unbalanced (say there are total 10 partitions and 2 pods. 8 partitions are assigned one of them and 2 partitions go to the other), it will be unbalanced forever because pods are kept healthy by k8s and no rebalancing ever occurs. And I don't think it is the desired behavior. My questions are 1. The desired behavior is partitions should be balanced eventually (but then it conflicts with "no rebalance" nature of static membership with the healthy backbone). Could you point out what I am missing here? 2. What is recommended under this unbalanced partition situation? 2-a Leaving it unbalanced? (unlikely) 2-b Do I have to adjust session timeout so I can artificially cause a rebalance that eventually makes assignment even? (I sense somewhat cumbersome) 2-c Is there a special PartitionAssignor implementation we should use under static membership and the assignor magically guarantees the assignment even? We've been suffering from rebalance storms from time to time and static membership seems the way to resolve it, but I do want to make sure we know how to work around some edge cases like it. -- -jun
