Hi Nicolas, an optimization like this would make a lot of sense. We did have some discussions around this already. However, it's more tricky to do than it seems at a first glace. We hope to introduce something like this for the next release.
-Matthias On 2/28/17 9:10 AM, Nicolas Fouché wrote: > Hi, > > I have 10 Kafka Streams processes which consume a topic with 10 partitions, > with a few changelog topics. > > Let's say these processes are all stopped, and I start them nearly at the > same time (in a matter of seconds). > > The first process seems to start initializing all state stores, which takes > 15 to 30 minutes in my case. In the meantime all other processes wait with > "(Re-)joining group xyz". > > When the first process finishes its initialization, everyone joins, and > everyone (including the first process) initializes state stores again. > > I wondered: what if the first process would wait a few minutes, to see if > others processes arrive in the group ? So no re-distribution would be > needed. > > - Nicolas >
signature.asc
Description: OpenPGP digital signature
