[ 
https://issues.apache.org/activemq/browse/AMQ-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=50234#action_50234
 ] 

Gary Tully commented on AMQ-2102:
---------------------------------

sorry, my patch is still a bit of a work in progress. I was using this jira as 
a shared holder. For the cce,{code} private QueueMessageReference 
getMatchingMessage(MessageDispatchNotification messageDispatchNotification) 
throws Exception {code} needs to return a QueueMessageReference. 
Add the following to get it to work better. I still need to validate that the 
dispatch that that happens here is correct w.r.t a normal dispatch. I have run 
it by applying the patch to trunk, so only with your changes to MasterBroker, 
ignoring the MasterBroker changes in the patch. With 30000 messages I still get 
errors, but of a different kind. More work needed. 
{code}
            if (message == null) {            
                synchronized (messages) {
                    try {
                        messages.reset();
                        while (messages.hasNext()) {
                            MessageReference node = messages.next();
+                            node.incrementReferenceCount();
+                            messages.remove();
                            if (messageId.equals(node.getMessageId())) {
+                                message = 
this.createMessageReference(node.getMessage());
{code}

> Master/slave out of sync with multiple consumers
> ------------------------------------------------
>
>                 Key: AMQ-2102
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2102
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.2.0
>            Reporter: Dan James
>            Assignee: Gary Tully
>         Attachments: master.xml, MasterSlaveBug.java, MasterSlavePatch.patch, 
> slave.xml, slaveDispatchOnNotification.patch
>
>
> I'm seeing exceptions like this in a simple master/slave setup:
> ERROR Service                        - Async error occurred: 
> javax.jms.JMSException: Slave broker out of sync with master: Dispatched 
> message (ID:DUL1SJAMES-L2-1231-1233929569359-0:4:1:1:207) was not in the 
> pending list for MasterSlaveBug
> javax.jms.JMSException: Slave broker out of sync with master: Dispatched 
> message (ID:DUL1SJAMES-L2-1231-1233929569359-0:4:1:1:207) was not in the 
> pending list for MasterSlaveBug
> The problem only happens when there are multiple consumers listening to the 
> queue, and is more likely to occur as there are more consumers listening.  
> I've written a test program that demonstrates the problem.
> I start the master and slave with an empty data directory and let them both 
> startup and settle.  Then start the test program.  The test program creates a 
> specified number of consumers, and then starts queuing 256 messages.  The 
> consumers process the message by sending a reply.  The producer counts the 
> replies.  Both consumers and the producer see all the messages, but with 
> multiple consumers it is very likely that the error above will occur and 
> several of the messages will still be queued on the slave.
> While debugging through the activemq code, I noticed that both the master and 
> the slave dispatch the message to a consumer's pending list independently.  
> In other words, it is possible that the master will add the message to 
> consumer A's pending list and the slave will add the message to consumer B's 
> pending list.  Once the message has been processed by consumer A, the master 
> sends a message to the slaving which specifies consumer A so that the slave 
> can remove the message.  The slave looks on its copy of consumer A's pending 
> list and cannot find the message.  As a result, it throws this exception and 
> the message stays stuck on consumer B's pending list on the slave.
> Master and slave configurations along with MasterSlaveBug.java are attached 
> to this issue.
> Start master and slave brokers:
> activemq xbean:master.xml
> activemq xbean:slave.xml
> Run with (only one consumer, the bug does not appear):
>    java -classpath .:activemq-all-5.2.0.jar MasterSlaveBug 1
> Run with (sixteen consumers, the bug does appear):
>    java -classpath .:activemq-all-5.2.0.jar MasterSlaveBug 16

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to