Re: Using 100's of ZK Observers

James, we treat observers not part of the ensemble's dynamic config, and it's all using -1 as server id, that's fine for us since we don't allow global sessions on observers. If you don't need global sessions on observers, then probably you can adopt similar solutions here for now. Thanks, Fangmi

Re: Using 100's of ZK Observers

Thanks Fangmin. That's an Interesting feature - allowing followers to host observers. but I assume the entire collection of servers is still considered part of the ensemble. If so, isn't the upper limit still capped to 256 - by the lowest 8 bits of the server id? On Fri, Apr 10, 2020 at 5:32 PM F

Re: Using 100's of ZK Observers

There is ObserverMaster feature contributed back in ZOOKEEPER-3140 could be used to scale the number of observers and traffics a single ensemble can support. It allows followers to serve observers as well, which relieves the fanout load on lea

Re: Using 100's of ZK Observers

If you have 100s of 1000s of ZK clients then having observer in each pod will presumably reduce traffic as most of the fan out traffic, from server to clients is localized to each pod. Observer is not part of quorum, and a quorum can't scale pass a few servers (typical just 5 or 7). Observers can

Re: Using 100's of ZK Observers

That was my instinct as well. I *think* any ZK writes would require a quorum before the transaction is committed. Getting a quorum over a several hundred/thousand node ensemble seems like a lot of traffic. Plus, from what I've read - though not 100% certain, it seems the number ZK nodes is capped a

Re: Using 100's of ZK Observers

On 10/04/2020 20:13, James Arbo wrote: > When we proposed this, there was great concern from the software architects > that network traffic between the kubernetes pods and the ZK ensemble must > be minimized. > This means that, at a minimum, we would be running at least 1 ZK ensemble > member on