[
https://issues.apache.org/jira/browse/ZOOKEEPER-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325223#comment-17325223
]
Li Wang commented on ZOOKEEPER-3140:
------------------------------------
Thanks for the great feature, [~enixon]
I am wondering if you have any performance data that you can share with the
community. Based on the comment, it looks like the performance benefits will be
seen when scaling an ensemble from hundreds to thousands of observers.
We have done some perf test with the following ensembles and didn't see much
difference.
1) 5 participants and 5 observers
2) 5 participants and 30 observers
It doesn't look like that there is a metrics for measuring the time that
observers finish sync and start serving client traffic. How do you think adding
one (observer_sync_time) just like what we have for follower (i.e.
follower_sync_time)
Thanks,
Li
> Allow Followers to host Observers
> ---------------------------------
>
> Key: ZOOKEEPER-3140
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3140
> Project: ZooKeeper
> Issue Type: New Feature
> Components: server
> Affects Versions: 3.6.0
> Reporter: Brian Nixon
> Assignee: Brian Nixon
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> Observers function simple as non-voting members of the ensemble, sharing the
> Learner interface with Followers and holding only a slightly difference
> internal pipeline. Both maintain connections along the quorum port with the
> Leader by which they learn of all new proposals on the ensemble.
>
> There are benefits to allowing Observers to connect to the Followers to plug
> into the commit stream in addition to connecting to the Leader. It shifts the
> burden of supporting Observers off the Leader and allow it to focus on
> coordinating the commit of writes. This means better performance when the
> Leader is under high load, particularly high network load such as can happen
> after a leader election when many Learners need to sync. It also reduces the
> total network connections maintained on the Leader when there are a high
> number of observers. One the other end, Observer availability is improved
> since it will take shorter time for a high number of Observers to finish
> syncing and start serving client traffic.
>
> The current implementation only supports scaling the number of Observers
> into the hundreds before performance begins to degrade. By opening up
> Followers to also host Observers, over a thousand observers can be hosted on
> a typical ensemble without major negative impact under both normal operation
> and during post-leader election sync.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
