Hi All,
Yesterday we did a hackathon on testing MB both as a stand alone and in
clustered setup to identify the stability. The following are the issues we
encountered and its status in relation to stand alone testing conducted.
Hasitha will be able to provide more insight on the cluster wide tests
carried out.
The following use cases were tested,
1) Testing growing queue count where having one publisher and a subscriber
for each queue and creating multiple queues, when the queue count started
to grow > 10 we observed the following,
[2014-10-06 12:41:55,687] INFO
{org.wso2.andes.subscription.SubscriptionStore} - New Local Subscription
Added [MBQ2]ID=
3@NodeQueue_NODEpuppet.wso2.com/127.0.0.1/D=true/X=false/O=null/E=amq.direct/ET=org.wso2.andes.server.exchange.DirectExchange$1@4a714747/EUD=0/S=true
Exception in thread "pool-31-thread-1"
java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
at java.util.ArrayList$Itr.next(ArrayList.java:831)
at
org.wso2.andes.server.slot.SlotDeliveryWorker.run(SlotDeliveryWorker.java:78)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
We identified the issues for the above (which was due to having a iterative
ArrayList which was modified in the QueueDeliveryWorker) Darshana worked on
a fix for it.
Also in the same test case we noticed the following when sending large
number of messages,
[2014-10-06 12:59:35,126] ERROR
{org.wso2.andes.server.cassandra.QueueDeliveryWorker} - Flusher queue is
growing, and this should not happen. Please check cassandra Flusher
I believe it was also identified by Sajini, she was working on a fix for
it. She will provide an update for it.
2) Testing through binding multiple subscribers to a particular queue and
having one publisher publishing to it (Many to One)
Bound upto 10 subscribers to a queue. published messages starting from
10-100000, all the messages were successfully delivered and distributed
among the subscribers. But noticed a performance decrees when the number of
subscribers incremented event if the publishing rate was the same.
3) Testing focusing on multiple subscribers and multiple publishers (Many
to Many)
Where subscriber count was incremented and in parallel the number of
publishers were incremented. The subscriber count was incremented upto 10
and the publisher count was incremented upto 5. In this case all the
messages were delivered successfully.
4) Testing having one subscriber and multiple publishers (One to Many)
In this case 10,000 messages were sent from each publisher and had one
subscriber receiving the messages overall the expected message count was
50,000 but we received only 40,000. Also the error which appeared in 1 st
case
[2014-10-06 12:59:35,126] ERROR
{org.wso2.andes.server.cassandra.QueueDeliveryWorker} - Flusher queue is
growing, and this should not happen. Please check cassandra Flusher
was noticed.
We opened up jiras for the relevant issues and in the process of fixing
them.
Thanks,
Pamod
--
*Pamod Sylvester *
*Senior Software Engineer *
Integration Technologies Team, WSO2 Inc.; http://wso2.com
email: [email protected] cell: +94 77 7779495
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
