Hi,
I agree that the time-out mechanism is not the ideal solution (depending on
number of messages to purge time needed would vary).
But per my understanding, your observation brings doubts.
- Copying from global queue to node queue happens faster than delivering
messages from node queue to subscribers. As we are running a dummy
subscriber for purging, according to above fact delivery time is only
limited upperly by "delivering to subscribers from node queue". Thus your
observation should be checked.
- We can find an alternative way to delete all messages addressed to
queues exposing the functionality as an MBean.
- Get messages addressed to queue being purged.
- Remove all metadata of those messages direcly from Cassandra
- Remove message content of those message IDS
- Remove message properties of those message IDs
- Remove message-queue mapping data of those message IDs.
- Finally, remove those message IDs as well.
This is done in a consistent way while deleting a queue. There we
Purge+Remove queue from qpid queue registry. Here what we have to do is
just delete all messages.
With this I have a doubt if purging works OK with clustered env.
As messages might be in different node queues, purging from one Mgt console
will not purge all.
Thus I think we need to go to above MBean solution. Then comes the
problem of security of operation. Anybody can invoke this without going
through Admin console even if we limit it through Mgt console.
Thanks.
On Sun, May 26, 2013 at 8:28 PM, Ishara Premadasa <[email protected]> wrote:
> Hi,
>
> When trying to fix [1] I discovered that the purging option for a queue
> performs low and it seems that this is due to a delay (or limitation) in
> moving messages to Subscriber queues from the Global Queues.
>
> With the present purging functionality the MessageConsumer waits on a
> given queue for the next message to arrive within a timeout interval of 5
> seconds [2] . However when trying to purge around 10,000 messages it seems
> that this timeout is not enough as it takes about 1 minute waiting time,
> when around 7000 messages are purged first and the Queue Purge client has
> to wait for 1 minute till the next 3000 messages are arrived to the Node
> queue.
>
> I carried out a test on the average MessageConsumer's timeout interval
> needed for successfully purging a queue with given messages count and
> results are as follows.
>
> *Message Count * *Timeout Interval Needed**
> *
>
> 7,500 5000 ms ( 5 s )
> 10,000 60000 ms ( 1 min )
> 15,000 90000 ms ( 1.5 min )
>
> As it takes a much longer timeout value for purging a message count like
> 10,000 i think this is unacceptable and makes a performance concern.
> Therefore we may need to find a way to make the message copying between the
> Global Queue --> Node Queue faster in order to overcome this.
>
> In trying to find out a solution, observed that the purging process
> becomes faster when adding a higher value into the following configuration
> in qpid-config.xml. I was able to purge a queue of 10000 messages within
> 5000 ms timeout interval after editing this entry.
>
> *<!--Message Batch size moved to Subscriber queues from the Global Queue
> at once. Increasing this values increase the chance of breaking in order
> delivery guarantee but performance will be improved-->
>
> <messageBatchSizeForSubscribersQueues>2000</messageBatchSizeForSubscribersQueues>
> *
>
> However as it says increasing this value also makes performance issues so
> this may not be a solution in the long run. In case i have missed anything,
> do we have any other restrictions in the Queue model which makes this slow
> nature? Or else please suggest what would be the suitable approach in
> handling this? (As this is not a bug in purging code but have to be handled
> with improving time between message copying into queues.)
>
> Thanks!
> Ishara
>
>
> [1] https://wso2.org/jira/browse/MB-234
> [2]
> http://docs.oracle.com/javaee/1.4/api/javax/jms/MessageConsumer.html#receive%28long%29
>
> --
> Ishara Premasada
> Software Engineer, Integrations TG
> WSO2 Inc. http://wso2.com/
> *Blog : http://isharapremadasa.blogspot.com/
> Twitter : https://twitter.com/ishadil
> Mobile : +94 714445832*
>
>
>
--
*Hasitha Abeykoon*
Software Engineer; WSO2, Inc.; http://wso2.com
*cell:* *+94 719363063*
*blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com>* *
*
*
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
