Thanks! ________________________________ From: Ken Giusti <[email protected]> Sent: Friday, September 29, 2017 3:54:20 PM To: users; [email protected] Subject: Re: Consumer affinity (JMSXGroupID), AMQP 1.0
At the moment the qpid dispatch router ignores all group related message properties when computing the route for a destination. I've opened a corresponding feature request against the router to track support for message groups: https://issues.apache.org/jira/browse/DISPATCH-843 On Fri, Sep 29, 2017 at 9:23 AM, Keith W <[email protected]> wrote: > A JIRA has been raised for this problem: > > https://issues.apache.org/jira/browse/QPID-7937 > > On 29 September 2017 at 11:47, Rob Godfrey <[email protected]> wrote: >> On 29 September 2017 at 10:35, Adel Boutros <[email protected]> wrote: >> >>> Hello Rob, >>> >>> >>> I would like to give my opinion on this. >>> >>> >>> In our current use cases, we are configuring the brokers dynamically using >>> the REST API. We would like to have this possibility in the use case of >>> Olivier. >> >> >>> Also, having to restart the virtual host is damaging the High Availability >>> of the messaging system. This is because while the Virtual Host is >>> restarting, no queues are available and the messages are inaccessible. So I >>> was wondering if restarting the virtual is a must or it could be fixed in a >>> Jira story? >>> >>> >> >> Sorry - my wording wasn't very clear. If you set this when you first >> create the queue then you don't need to restart the vhost... however if you >> have an existing queue that you want to change, then the effect won't be >> seen until the vhost is restarted (basically the queue sets itself up to be >> "group aware" or "not group aware" when it starts up, it can't change while >> it is running). I don't think this is really an issue for you in that when >> you create your queues with the REST API you just need to set this >> attribute. >> >> >>> >>> Regarding the 2nd point of multiple brokers and dispatch router, if all >>> brokers have the same queues configured with the appropriate grouping >>> config, would that really break the feature? >>> >>> In our current use cases, all components have the same configuration all >>> the time (All brokers have same queues and same for the dispatch router). >>> >>> So I would imagine if a consumer wants to consume a message with the >>> correct group, the dispatch router will propagate this header to any >>> available broker and will only get the corresponding messages. >>> >>> >>> >> The way grouping works is that the first time a message with a particular >> group id comes along, the broker will assign that group to a particular >> consumer. Each broker will do this independently. So if you have two >> brokers B1 and B2; and two consumers C1 and C2 and message M of group A >> arrives on B1 whereas message N of group A arrives on B2; then B1 may >> decide to associate group A with C1 whereas broker B2 decides to associate >> group A with consumer C2. So message groups with multiple brokers will not >> work behind a router *unless* the router is enhanced so that it is aware of >> message grouping (so all messages of the same group are sent to the same >> broker) or the brokers somehow share information about the group -> >> consumer assignment. >> >> -- Rob >> >> What do you think? >>> >>> >>> Regards, >>> >>> Adel >>> >>> ________________________________ >>> From: Rob Godfrey <[email protected]> >>> Sent: Friday, September 29, 2017 9:14:52 AM >>> To: [email protected] >>> Subject: Re: Consumer affinity (JMSXGroupID), AMQP 1.0 >>> >>> On 29 September 2017 at 01:09, Olivier Mallassi < >>> [email protected]> >>> wrote: >>> >>> > Gentleman, >>> > >>> > I will need your help. >>> > >>> > I have a use case where I would like to guarantee "consumer affinity", >>> > which is usually implemented using the JMSXGroupID (actually, I am sure >>> it >>> > was working w/ AMQP 0.10 clients but here I am using AMQP 1.0) >>> > >>> > My test does the "classical" case: >>> > for (i ... i < 100.) >>> > Message textMsg = new TextM.... >>> > textMsg.setStringProperty("JMSXGroupID", "groupA") >>> > MessageProducer.send(textMsg); >>> > >>> > and I start two consumers (assuming only one would work). >>> > >>> > What I observe is that both consumers are receiving messages, acting as >>> > competing consumers. I also tried added the qpid.group_header_key >>> property >>> > to the queue but it does not change anything. >>> > >>> > >>> > My setup is >>> > - java broker 6.0.4 & 6.1.4 >>> > - java clients using qpid-jms 0.25.0 (AMQP 1.0) >>> > >>> > Is this the expected behaviour? Any ideas? >>> > >>> > >>> This is a bug in the AMQP 1.0 behaviour of the broker. The Message >>> Grouping functionality was originally built around the AMQP 0-x protocols >>> and is looking for a message group in the application headers of the >>> message. In AMQP 1.0 there is a dedicated property for this and the JMS >>> client is (correctly from an AMQP 1.0 point of view) placing the group id >>> there. >>> >>> As a work around instead of using JMSXGroupID you could use any other (non >>> JMSX prefixed) header, e.g.: >>> >>> textMsg.setStringProperty("qpidBugWorkaround", "groupA") >>> >>> Note that on the broker you need to configure the queue so it knows which >>> header to use; this is stored in the messageGroupKey property of the queue >>> (to "qpidBugWorkaround" in this case) . Note that after setting this >>> property you will need to restart the virtual host before the change takes >>> effect. >>> >>> >>> > >>> > Complementary question: let's assume now, that there is the DR between >>> the >>> > broker & the consumers. Does the DR "propagate" or support "grouping >>> > messages"? (I assume it is depending if you route the link or messages) >>> > >>> >>> With a single broker and link routing, yes grouping messages should >>> continue to work. >>> >>> If there are multiple brokers sharding the queue then it would not work - >>> to support this the router would need to be changed to recognize grouping >>> and send all messages with the same group id to the same broker waypoint >>> (also, in this case, you would need to use message routing for inbound >>> messages and link routing for consuming messages). We have talked about >>> support for behaviour like this before, but there is nothing implemented >>> yet as far as I know. >>> >>> -- Rob >>> >>> >>> > >>> > >>> > thanks for your help >>> > >>> > Regards. >>> > >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > -- -K --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
