On Sun, Nov 24, 2013 at 9:57 PM, Sznajder ForMailingList <[email protected]> wrote: > Hi, > > I share with you a very problematic (for me...) issue, I encountered. > > I wrote a very simple Queue (Producer and consumer). > > The producer, put to the ZK Queue items (integer) when the queue reached > half of its initial size. > And Consumer just picks from the list and prints to a log file the consumed > item and sleeps some time. > > In addition, I defined the Consumer to be multi-threaded > > I tested the following configurations on different quorums (when each server > is 16 cores) > > Quorum of 2 servers : One consumer and one producer, 2 threads for the > consumer: > In 5 minutes, I consumed 1320 items. > > When I make the consumer to be 16 threads, I get: 10890 items consumed. All > is OK. > > > > > However, when I move to a quorum of three, with 2 consumers (and each > consumer 16 threads), I suddenly get: > In 5 minutes 12634 items consumed. That means, that adding a new consumer > did not enhance the throughput of the system. > > > I am blocked with this issue, several days, and any help will be welcome!! > > Benjamin >
Hello Benjamin, Did you measure add/delete thru-put for your cluster? If not, can you check if it is significantly higher than 12k? This should help to get you started: https://github.com/brownsys/zookeeper-benchmark Regards, Vamsi
