Re: Kafka Stretch Clusters

2022-05-12 Thread Guozhang Wang
> Am I correct in assuming that if the preferred leader is not available, the next replica in the ISR list is chosen to be the leader? Yes, that's correct :) On Wed, May 11, 2022 at 1:15 PM Andrew Otto wrote: > Thanks so much Guozhang! > > > 1) For the producer -> leader hop, could you save th

Re: Kafka Stretch Clusters

2022-05-11 Thread Andrew Otto
Thanks so much Guozhang! > 1) For the producer -> leader hop, could you save the cross-DC network? > even if your message's partition has to be determined deterministically by the key, in operations you can still see if most of your active producers are from one DC, then configure your topic part

Re: Kafka Stretch Clusters

2022-05-09 Thread Guozhang Wang
Hello Andrew. Just to answer your questions first, yes that's correct in your described settings that three round-trips between DCs would incur, but since the replica fetches can be done in parallel, the latency is not a sum of all the round-trips. But if you stay with 2 DCs only, the number of ro

Kafka Stretch Clusters

2022-05-09 Thread Andrew Otto
Hi all, I'm evaluating the feasibility of setting up a cross datacenter Kafka 'stretch' cluster at The Wikimedia Foundation. I've found docs here and there, but they are pretty slim. My biggest concern is the fact that while Follower Fetching