Hi Vamsi A short update, it appeared that Using lockPath() (from Apache curator) hurts the throuhput a lot. I wrote to the relevant mailing list a description of the issue.
best regards Benjamin On Mon, Nov 25, 2013 at 8:43 AM, Vamsi Devaki <[email protected]>wrote: > On Sun, Nov 24, 2013 at 9:57 PM, Sznajder ForMailingList > <[email protected]> wrote: > > Hi, > > > > I share with you a very problematic (for me...) issue, I encountered. > > > > I wrote a very simple Queue (Producer and consumer). > > > > The producer, put to the ZK Queue items (integer) when the queue reached > > half of its initial size. > > And Consumer just picks from the list and prints to a log file the > consumed > > item and sleeps some time. > > > > In addition, I defined the Consumer to be multi-threaded > > > > I tested the following configurations on different quorums (when each > server > > is 16 cores) > > > > Quorum of 2 servers : One consumer and one producer, 2 threads for the > > consumer: > > In 5 minutes, I consumed 1320 items. > > > > When I make the consumer to be 16 threads, I get: 10890 items consumed. > All > > is OK. > > > > > > > > > > However, when I move to a quorum of three, with 2 consumers (and each > > consumer 16 threads), I suddenly get: > > In 5 minutes 12634 items consumed. That means, that adding a new consumer > > did not enhance the throughput of the system. > > > > > > I am blocked with this issue, several days, and any help will be > welcome!! > > > > Benjamin > > > > Hello Benjamin, > > Did you measure add/delete thru-put for your cluster? If not, can you > check if it is significantly higher than 12k? > > This should help to get you started: > https://github.com/brownsys/zookeeper-benchmark > > Regards, > Vamsi >
