Hi Ishara, What is the purge? does that empty the queue? How is that implemented?
I think it is OK to purge operation to be async. Basically, if you say purge, it will start it in the background and delete them. I think if we can avoid message delivery after the purge operation, then it is OK. --Srinath On Sun, May 26, 2013 at 11:45 PM, Hasitha Hiranya <[email protected]> wrote: > Hi, > > I agree that the time-out mechanism is not the ideal solution (depending > on number of messages to purge time needed would vary). > But per my understanding, your observation brings doubts. > > > - Copying from global queue to node queue happens faster than > delivering messages from node queue to subscribers. As we are running a > dummy subscriber for purging, according to above fact delivery time is only > limited upperly by "delivering to subscribers from node queue". Thus your > observation should be checked. > - We can find an alternative way to delete all messages addressed to > queues exposing the functionality as an MBean. > - Get messages addressed to queue being purged. > - Remove all metadata of those messages direcly from Cassandra > - Remove message content of those message IDS > - Remove message properties of those message IDs > - Remove message-queue mapping data of those message IDs. > - Finally, remove those message IDs as well. > > This is done in a consistent way while deleting a queue. There we > Purge+Remove queue from qpid queue registry. Here what we have to do is > just delete all messages. > > With this I have a doubt if purging works OK with clustered env. > As messages might be in different node queues, purging from one Mgt console > will not purge all. > > Thus I think we need to go to above MBean solution. Then comes the > problem of security of operation. Anybody can invoke this without going > through Admin console even if we limit it through Mgt console. > > Thanks. > > > > > > On Sun, May 26, 2013 at 8:28 PM, Ishara Premadasa <[email protected]> wrote: > >> Hi, >> >> When trying to fix [1] I discovered that the purging option for a queue >> performs low and it seems that this is due to a delay (or limitation) in >> moving messages to Subscriber queues from the Global Queues. >> >> With the present purging functionality the MessageConsumer waits on a >> given queue for the next message to arrive within a timeout interval of 5 >> seconds [2] . However when trying to purge around 10,000 messages it seems >> that this timeout is not enough as it takes about 1 minute waiting time, >> when around 7000 messages are purged first and the Queue Purge client has >> to wait for 1 minute till the next 3000 messages are arrived to the Node >> queue. >> >> I carried out a test on the average MessageConsumer's timeout interval >> needed for successfully purging a queue with given messages count and >> results are as follows. >> >> *Message Count * *Timeout Interval Needed** >> * >> >> 7,500 5000 ms ( 5 s ) >> 10,000 60000 ms ( 1 min ) >> 15,000 90000 ms ( 1.5 min ) >> >> As it takes a much longer timeout value for purging a message count like >> 10,000 i think this is unacceptable and makes a performance concern. >> Therefore we may need to find a way to make the message copying between the >> Global Queue --> Node Queue faster in order to overcome this. >> >> In trying to find out a solution, observed that the purging process >> becomes faster when adding a higher value into the following configuration >> in qpid-config.xml. I was able to purge a queue of 10000 messages within >> 5000 ms timeout interval after editing this entry. >> >> *<!--Message Batch size moved to Subscriber queues from the Global >> Queue at once. Increasing this values increase the chance of breaking in >> order delivery guarantee but performance will be improved--> >> >> <messageBatchSizeForSubscribersQueues>2000</messageBatchSizeForSubscribersQueues> >> * >> >> However as it says increasing this value also makes performance issues so >> this may not be a solution in the long run. In case i have missed anything, >> do we have any other restrictions in the Queue model which makes this slow >> nature? Or else please suggest what would be the suitable approach in >> handling this? (As this is not a bug in purging code but have to be handled >> with improving time between message copying into queues.) >> >> Thanks! >> Ishara >> >> >> [1] https://wso2.org/jira/browse/MB-234 >> [2] >> http://docs.oracle.com/javaee/1.4/api/javax/jms/MessageConsumer.html#receive%28long%29 >> >> -- >> Ishara Premasada >> Software Engineer, Integrations TG >> WSO2 Inc. http://wso2.com/ >> *Blog : http://isharapremadasa.blogspot.com/ >> Twitter : https://twitter.com/ishadil >> Mobile : +94 714445832* >> >> >> > > > -- > *Hasitha Abeykoon* > Software Engineer; WSO2, Inc.; http://wso2.com > *cell:* *+94 719363063* > *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com>* * > * > * > -- ============================ Srinath Perera, Ph.D. Senior Software Architect, WSO2 Inc. Visiting Faculty, University of Moratuwa Member, Apache Software Foundation Research Scientist, Lanka Software Foundation Blog: http://srinathsview.blogspot.com/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
