They seem fair enough and quite related.
As a side note, I have a bug with the dispatch router 0.6.1 but I haven't
submitted it yet because I haven't reduced the test case yet.
In resume, when I connect 2 dispatchers (inter-router) and then delete the
connector/listener of "inter-router". If I delete and recreate a mobile address
which has received a message on one of the dispatchers, the stats of the "in"
and "out" do not reset to 0 when doing "qdstat -a" but they remain at the old
values. However they reset correctly on the other router.
Have you encountered something similar? Once I have a reduced test case, I will
post it in a different thread of course.
Regards,
Adel
________________________________
From: Ted Ross <tr...@redhat.com>
Sent: Thursday, September 29, 2016 4:38:26 PM
To: users@qpid.apache.org
Subject: Re: Testing failover on dispatcher/java-broker cluster
Sorry, those Jira numbers and descriptions are mismatched. Here's the
correct list:
- DISPATCH-496 - Activation of an autolink does not result in issuing
credit to a blocked sender
- DISPATCH-505 - Eventual loss of credit on inter-router control
links when the topology changes
- DISPATCH-523 - Topology changes can cause in-flight deliveries to
be stuck in the ingress router
On 09/29/2016 10:35 AM, Ted Ross wrote:
>
> On 09/24/2016 05:32 AM, Adel Boutros wrote:
>> We are indeed in favor of a minor release as long as the latest
>> version is still 0.6.x and we are willing to re-launch our tests and
>> give feedback on the release candidate once provided (It shouldn't
>> take us more than a day to compile and test).
>> Do you have a list of fixes in mind?
>
> I've identified three fixes that look like good candidates for 0.6.2:
>
> - DISPATCH-496 - Topology changes can cause in-flight deliveries to
> be stuck in the ingress router
> - DISPATCH-505 - Eventual loss of credit on inter-router control
> links when the topology changes
> - DISPATCH-523 - Activation of an autolink does not result in issuing
> credit to a blocked sender
>
> These are all stability-related issues.
>
> Thoughts?
>
> -Ted
>
>> Regards,Adel
>>
>>> Subject: Re: Testing failover on dispatcher/java-broker cluster
>>> To: users@qpid.apache.org
>>> From: tr...@redhat.com
>>> Date: Fri, 23 Sep 2016 17:23:57 -0400
>>>
>>> Hi Adel,
>>>
>>> A minor release is always possible. It's up to us, the community, to
>>> decide whether and when to produce one. I'm in favor of releasing an
>>> 0.6.2 with some small backports to fix bugs for users that want to stay
>>> on Proton 0.12.
>>>
>>> -Ted
>>>
>>> On 09/23/2016 09:44 AM, Adel Boutros wrote:
>>>> Hello Ted,
>>>> Did you happen to have the time to check if a minor release is
>>>> possible?
>>>> Regards,Adel
>>>>
>>>>> From: adelbout...@live.com
>>>>> To: users@qpid.apache.org
>>>>> Subject: RE: Testing failover on dispatcher/java-broker cluster
>>>>> Date: Tue, 20 Sep 2016 15:13:03 +0200
>>>>>
>>>>> Hello Ted,
>>>>>
>>>>> I confirm the fix solved the issue.
>>>>>
>>>>> Would it be possible to do a 0.6.2 release? We cannot compile newer
>>>>> versions of Proton (We currently use 0.12.2) due to lack of
>>>>> resources from our side and we really need this fix for our tests.
>>>>>
>>>>> Regards,
>>>>> Adel
>>>>>
>>>>>> Subject: Re: Testing failover on dispatcher/java-broker cluster
>>>>>> To: users@qpid.apache.org
>>>>>> From: tr...@redhat.com
>>>>>> Date: Mon, 19 Sep 2016 12:18:23 -0400
>>>>>>
>>>>>> Hi Adel,
>>>>>>
>>>>>> It's a one-liner and it applies cleanly to the 0.6.x branch.
>>>>>>
>>>>>> https://git-wip-us.apache.org/repos/asf?p=qpid-dispatch.git;h=41b7407
>>>>>>
>>>>>> -Ted
>>>>>>
>>>>>>
>>>>>> On 09/19/2016 11:41 AM, Adel Boutros wrote:
>>>>>>> Hello Ted,
>>>>>>>
>>>>>>> Antoine is on vacation so I will be taking over this task.
>>>>>>>
>>>>>>> Does this fix have any dependencies? We would like to apply it on
>>>>>>> 0.6.1 without other fixes because it seems the master branch
>>>>>>> requires proton 0.13.0 minimum whereas we have currently 0.12.2
>>>>>>> and we cannot upgrade at the time being.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Adel
>>>>>>>
>>>>>>>> Subject: Re: Testing failover on dispatcher/java-broker cluster
>>>>>>>> To: users@qpid.apache.org
>>>>>>>> From: tr...@redhat.com
>>>>>>>> Date: Fri, 16 Sep 2016 16:53:05 -0400
>>>>>>>>
>>>>>>>> Antoine,
>>>>>>>>
>>>>>>>> I think I know what that problem is. I belileve you've stumbled
>>>>>>>> upon
>>>>>>>> this issue:
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/DISPATCH-496
>>>>>>>>
>>>>>>>> Your second delivery, the one resulting in a timeout, is causing
>>>>>>>> the
>>>>>>>> inbound link to be blocked (i.e. it has undelivered messages).
>>>>>>>> When the
>>>>>>>> broker reattaches, the blocked links are supposed to become
>>>>>>>> unblocked
>>>>>>>> but they don't in the case of auto-links.
>>>>>>>>
>>>>>>>> This has been fixed on the master branch if you'd like to try
>>>>>>>> applying
>>>>>>>> the patch.
>>>>>>>>
>>>>>>>> -Ted
>>>>>>>>
>>>>>>>> On 09/15/2016 04:56 AM, Antoine Chevin wrote:
>>>>>>>>> Hi Ted,
>>>>>>>>>
>>>>>>>>> You’re right, the connection close looked strange before
>>>>>>>>> stopping of the
>>>>>>>>> broker. I manually added the annotation (# stopping the broker)
>>>>>>>>> and was
>>>>>>>>> wrong about the position of this one. I replayed the test and the
>>>>>>>>> connection close happens *after* the broker stop. I assume it
>>>>>>>>> is the broker
>>>>>>>>> that initiates it.
>>>>>>>>>
>>>>>>>>> I found something interesting. In my test, I always sent a
>>>>>>>>> message when the
>>>>>>>>> broker is down, expecting to get a JmsSendTimedOutException
>>>>>>>>> (waiting for
>>>>>>>>> the disposition frame). I assumed this was harmless. But it
>>>>>>>>> turns out this
>>>>>>>>> is not. When I don’t do that, I can send a message after the
>>>>>>>>> broker
>>>>>>>>> restart. So to sum up the experiment I did:
>>>>>>>>>
>>>>>>>>> * I use Wireshark between the JMS client and the dispatcher. *
>>>>>>>>>
>>>>>>>>> 1) Using JMS I establish a connection to the dispatcher
>>>>>>>>> and create a
>>>>>>>>> message producer (Wireshark: connection open -> attach)
>>>>>>>>> 2) I’m able to send a message to the broker through the
>>>>>>>>> dispatcher (
>>>>>>>>> Wireshark: transfer -> disposition)
>>>>>>>>> 3) I stop the broker
>>>>>>>>> 4) With the same link, I send a message and I get a
>>>>>>>>> JmsSendTimedOutException (waiting for the disposition frame)
>>>>>>>>> (Wireshark:
>>>>>>>>> transfer)
>>>>>>>>> 5) I restart the broker
>>>>>>>>> 6) With the same link, I try to send a message and I get a
>>>>>>>>> JmsSendTimedOutException for the same reason (waiting for the
>>>>>>>>> disposition
>>>>>>>>> frame) (Wireshark: transfer)
>>>>>>>>>
>>>>>>>>> If I skip step (4), I cannot reproduce step (6) and my messages
>>>>>>>>> arrive
>>>>>>>>> (Wireshark: transfer -> disposition) to the restarted broker.
>>>>>>>>>
>>>>>>>>> I hope it makes it clearer for you. Sorry for my rookie
>>>>>>>>> mistakes :-).
>>>>>>>>>
>>>>>>>>> Note: My colleague and I ran a small experiment to identify if
>>>>>>>>> the problem
>>>>>>>>> comes from JMS or the AMQP protocol. He changed the code of the
>>>>>>>>> java broker
>>>>>>>>> to not send the disposition frame one time out of two.
>>>>>>>>>
>>>>>>>>> We got these results:
>>>>>>>>>
>>>>>>>>> * I use Wireshark between the JMS client and the patched broker. *
>>>>>>>>>
>>>>>>>>> 1) Using JMS I establish a connection to the patched broker and
>>>>>>>>> create a
>>>>>>>>> message producer (Wireshark: connection open -> attach)
>>>>>>>>> 2) I send a message to the broker and it replies with the
>>>>>>>>> disposition
>>>>>>>>> frame (Wireshark: transfer -> disposition)
>>>>>>>>> 3) I send a message to the broker which drops the disposition
>>>>>>>>> frame. I get
>>>>>>>>> a send timeout in JMS (Wireshark: transfer)
>>>>>>>>> 2) I send a message to the broker and it replies with the
>>>>>>>>> disposition frame
>>>>>>>>> (Wireshark: transfer -> disposition). It works fine.
>>>>>>>>>
>>>>>>>>> We assume that there is something going on in the dispatcher.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Antoine
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
>>>>>>>> For additional commands, e-mail: users-h...@qpid.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
>>>>>> For additional commands, e-mail: users-h...@qpid.apache.org
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
>>> For additional commands, e-mail: users-h...@qpid.apache.org
>>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
