This doesn’t surprise me. The Javadoc for the lockPath() method even says "There is a small performance penalty for this behavior however.”. I guess we should update that, but taking a lock for every message will incur a performance penalty.
-Jordan On Nov 28, 2013, at 4:48 AM, Sznajder ForMailingList <[email protected]> wrote: > Hi > > Following my precedent questions on this mailing list, I continued to > investigate and found that using lockPath() in the QueueBuilder can hurt the > throughput of the consumption of the queue items by 134%. > > In fact the lockPath() starts to seriously hurting the throughput when we > multiply the number of queue consumers. > > > For helping you to reproduce the problem, I wrote the attached code. > > The main class gets one parameter : the connectString to the ZQ quorum. If no > parameters is provided, the class will launch the Curator TestingServer. > > > The producer starts by putting 100 items in the queue, then it monitors the > number of elements in the queue and measures the time spent to consume the > whole queue. > > The consumers just consume the items without doing anything else, and then > sleep 500 msecs. > > I tested the program with these different parameters: > 1, 2, 4, 8, 16 and 32 consumers... > with setting the lockPath() and without setting the lockPath(). > > > > And I get the number described in the attached Excel file. You can see that > for 1 or 2 consumers, the hurt by using lockPath() isn't so significant, but > it can reach 134% when we use a larger number of consumers.... > > > I put here the output: > > [java] Without LOCK_PATH > [java] -=-=-=-=-=-= 1 consumers =-=-=-=-=-=- > [java] It took 29562 msecs for consuming 100 items > [java] -=-=-=-=-=-= 32 consumers =-=-=-=-=-=- > [java] It took 1559 msecs for consuming 100 items > [java] -=-=-=-=-=-= 16 consumers =-=-=-=-=-=- > [java] It took 2022 msecs for consuming 100 items > [java] -=-=-=-=-=-= 8 consumers =-=-=-=-=-=- > [java] It took 4045 msecs for consuming 100 items > [java] -=-=-=-=-=-= 4 consumers =-=-=-=-=-=- > [java] It took 7541 msecs for consuming 100 items > [java] -=-=-=-=-=-= 2 consumers =-=-=-=-=-=- > [java] It took 15539 msecs for consuming 100 items > [java] > [java] > [java] > [java] With LOCK_PATH > [java] -=-=-=-=-=-= 1 consumers =-=-=-=-=-=- > [java] It took 30562 msecs for consuming 100 items > [java] -=-=-=-=-=-= 32 consumers =-=-=-=-=-=- > [java] It took 3639 msecs for consuming 100 items > [java] -=-=-=-=-=-= 16 consumers =-=-=-=-=-=- > [java] It took 4610 msecs for consuming 100 items > [java] -=-=-=-=-=-= 8 consumers =-=-=-=-=-=- > [java] It took 6101 msecs for consuming 100 items > [java] -=-=-=-=-=-= 4 consumers =-=-=-=-=-=- > [java] It took 9073 msecs for consuming 100 items > [java] -=-=-=-=-=-= 2 consumers =-=-=-=-=-=- > [java] It took 16053 msecs for consuming 100 items > > <QueueConsumerProducer.java><SimpleQueueItem.java><Throughput.pdf>
