Camille, that's a very good question. Largest cluster I've heard about
is 10k sessions.
Jeremy - largest I've ever tested was a 3 server cluster with ~500
sessions. Each session created 10k znodes (100bytes each znode) and
set 5 watches on each. So 5 million znodes and 25million watches. I
then had the sessions delete the znodes and looked for the
notifications. They were processed by the clients quite quickly (order
of seconds) iirc. Note: this required some GC tuning on the servers to
operate correctly (in particular cms and incremental gc was turned on
and sufficient memory was allocated for the heaps).
here's a similar test setup I used:
this is the latency tester tool
On Thu, Nov 18, 2010 at 9:44 AM, Fournier, Camille F. [Tech]
> Can you clarify what you mean when you say 10-100K watchers? Do you mean
> 10-100K clients with 1 active watch, or some lesser number of clients with
> more watches, or a few clients doing a lot of watches and other clients doing
> other things?
> -----Original Message-----
> From: Jeremy Hanna [mailto:jeremy.hanna1...@gmail.com]
> Sent: Thursday, November 18, 2010 12:15 PM
> To: firstname.lastname@example.org
> Subject: number of clients/watchers
> I had a question about number of clients against a zookeeper cluster. I was
> looking at having between 10,000 and 100,000 (towards 100,000) watchers
> within a single datacenter at a given time. Assuming that some fraction of
> that number are active clients and the r/w ratio is well within the zookeeper
> norms, is that number within the realm of possibility for zookeeper? We're
> going to do testing and benchmarking and things, but I didn't want to go down
> a rabbit hole if this is simply too much for a single zookeeper cluster to
> handle. The numbers I've seen in blog posts vary and I saw that the
> observers feature may be useful in this kind of setting.
> Maybe I'm underestimating zookeeper or maybe I don't have enough information
> to tell. I'm just trying to see if zookeeper is a good fit for our use case.