AMQP 1.0 and Shared Subscriptions
When we were implementing the MQ Light broker, we wanted to be able to support sharing of subscriptions across a group of clients - in particular for the worker offload scenario. At the time we were unable to find guidance in the specification or extensions thereof, so we went ahead with encoding this into the terminus addresses of an attach. Using a `private:` prefix to denote an exclusive subscription, and `share:sharename` to indicate a shared one. i.e., - @attach(18) [name=share:sharename:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] vs - @attach(18) [name=private:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=private:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=private:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] The other day I happened to notice that the qpid-cpp broker also supports a similar functionality since https://issues.apache.org/jira/browse/QPID-4917 / https://github.com/apache/qpid/commit/d4dafd3 - and it does this by supporting a capability of 'shared' on the exchange, and expecting a client @attach to request this capability, whereupon the link-name is used as-is for the name of the share. I wasn't able to find this behaviour documented any further than that, and it wasn't clear to me what the behaviour should be for the various scenarios. Ideally we'd like to conform to a common standard. Does anyone know if there were any plans to register this (or another) shared subscription behaviour with the working group as an extension? -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: AMQP 1.0 and Shared Subscriptions
Bump this up. I'm essentially in the same boat. The olso.messaging library in Openstack has introduced a feature called 'consumer pools' which is essentially the same pattern: a set of subscribers to a topic can belong to a pool. Each distinct pool of subscribers get a copy of the message sent to the topic, and only one subscriber from the pool claims the message. This pattern is available in the JMS 2.0 API via the createdSharedConsumer call [1] I'm particularly interested in how such a feature could be implemented in a qpid-dispatch based federation of brokers. I suspect that all subscribers to a given share would have to belong on the same broker - if shares existed on multiple brokers then each 'sub-share' would get a copy of the same message in violation of the pattern. [1] http://www.oracle.com/technetwork/articles/java/jms2messaging-1954190.html - Original Message - From: Dominic Evans dominic.ev...@uk.ibm.com To: proton@qpid.apache.org Sent: Wednesday, May 6, 2015 7:41:59 AM Subject: AMQP 1.0 and Shared Subscriptions When we were implementing the MQ Light broker, we wanted to be able to support sharing of subscriptions across a group of clients - in particular for the worker offload scenario. At the time we were unable to find guidance in the specification or extensions thereof, so we went ahead with encoding this into the terminus addresses of an attach. Using a `private:` prefix to denote an exclusive subscription, and `share:sharename` to indicate a shared one. i.e., - @attach(18) [name=share:sharename:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] vs - @attach(18) [name=private:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=private:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=private:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] The other day I happened to notice that the qpid-cpp broker also supports a similar functionality since https://issues.apache.org/jira/browse/QPID-4917 / https://github.com/apache/qpid/commit/d4dafd3 - and it does this by supporting a capability of 'shared' on the exchange, and expecting a client @attach to request this capability, whereupon the link-name is used as-is for the name of the share. I wasn't able to find this behaviour documented any further than that, and it wasn't clear to me what the behaviour should be for the various scenarios. Ideally we'd like to conform to a common standard. Does anyone know if there were any plans to register this (or another) shared subscription behaviour with the working group as an extension? -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- -K
Re: AMQP 1.0 and Shared Subscriptions
On 05/06/2015 10:05 AM, Ken Giusti wrote: Bump this up. I'm essentially in the same boat. The olso.messaging library in Openstack has introduced a feature called 'consumer pools' which is essentially the same pattern: a set of subscribers to a topic can belong to a pool. Each distinct pool of subscribers get a copy of the message sent to the topic, and only one subscriber from the pool claims the message. This pattern is available in the JMS 2.0 API via the createdSharedConsumer call [1] I'm particularly interested in how such a feature could be implemented in a qpid-dispatch based federation of brokers. I suspect that all subscribers to a given share would have to belong on the same broker - if shares existed on multiple brokers then each 'sub-share' would get a copy of the same message in violation of the pattern. A half-baked idea: What if we implemented a shared-target in Dispatch? It's like a waypoint except it doesn't have any external component and has no connections or links associated with it. You would configure a shared-target to receive from topic address T and send to pool address P. The semantics for T are multicast (i.e. each pool receiving from T will get one copy of every message from T) and the semantics for P are anycast (i.e. messages to P will be sent to exactly one of the consumers of P). Would that solve your problem? [1] http://www.oracle.com/technetwork/articles/java/jms2messaging-1954190.html - Original Message - From: Dominic Evans dominic.ev...@uk.ibm.com To: proton@qpid.apache.org Sent: Wednesday, May 6, 2015 7:41:59 AM Subject: AMQP 1.0 and Shared Subscriptions When we were implementing the MQ Light broker, we wanted to be able to support sharing of subscriptions across a group of clients - in particular for the worker offload scenario. At the time we were unable to find guidance in the specification or extensions thereof, so we went ahead with encoding this into the terminus addresses of an attach. Using a `private:` prefix to denote an exclusive subscription, and `share:sharename` to indicate a shared one. i.e., - @attach(18) [name=share:sharename:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] vs - @attach(18) [name=private:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=private:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=private:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] The other day I happened to notice that the qpid-cpp broker also supports a similar functionality since https://issues.apache.org/jira/browse/QPID-4917 / https://github.com/apache/qpid/commit/d4dafd3 - and it does this by supporting a capability of 'shared' on the exchange, and expecting a client @attach to request this capability, whereupon the link-name is used as-is for the name of the share. I wasn't able to find this behaviour documented any further than that, and it wasn't clear to me what the behaviour should be for the various scenarios. Ideally we'd like to conform to a common standard. Does anyone know if there were any plans to register this (or another) shared subscription behaviour with the working group as an extension? -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: AMQP 1.0 and Shared Subscriptions
On some recent traces to an ActiveMQ 5.12 broker I observed that the broker
signals some info in the open frame:
capability:ANONYMOUS-RELAY,
property:{topic-prefix:topic://, queue-prefix:queue://}
Having written clients that try to go to ActiveMQ brokers and to Qpid brokers,
getting the queue and topic prefixes from the broker is extremely convenient as
long as the broker requires a prefix. If ActiveMQ needs a prefix, Qpidd uses a
node property, and MQ Light uses different prefixes then interoperability
becomes an issue. I'd much rather see all brokers use the property scheme and
not overload the address with prefixes.
- Original Message -
From: Gordon Sim g...@redhat.com
To: us...@qpid.apache.org
Cc: proton@qpid.apache.org
Sent: Wednesday, May 6, 2015 10:54:26 AM
Subject: Re: AMQP 1.0 and Shared Subscriptions
Moving to the user list as this is a more general topic.
On 05/06/2015 12:41 PM, Dominic Evans wrote:
When we were implementing the MQ Light broker, we wanted to be able
to support sharing of subscriptions across a group of clients - in
particular for the worker offload scenario. At the time we were
unable to find guidance in the specification or extensions thereof,
so we went ahead with encoding this into the terminus addresses of an
attach. Using a `private:` prefix to denote an exclusive
subscription, and `share:sharename` to indicate a shared one.
i.e.,
- @attach(18) [name=share:sharename:topicname, handle=0,
role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40)
[address=share:sharename:topicname, durable=0, timeout=0,
dynamic=false], target=@target(41)
[address=share:sharename:topicname, durable=0, timeout=0,
dynamic=false], initial-delivery-count=0]
vs
- @attach(18) [name=private:topicname, handle=0, role=true,
snd-settle-mode=0, rcv-settle-mode=0, source=@source(40)
[address=private:topicname, durable=0, timeout=0, dynamic=false],
target=@target(41) [address=private:topicname, durable=0,
timeout=0, dynamic=false], initial-delivery-count=0]
The other day I happened to notice that the qpid-cpp broker also
supports a similar functionality since
https://issues.apache.org/jira/browse/QPID-4917 /
https://github.com/apache/qpid/commit/d4dafd3 - and it does this by
supporting a capability of 'shared' on the exchange, and expecting a
client @attach to request this capability, whereupon the link-name is
used as-is for the name of the share. I wasn't able to find this
behaviour documented any further than that, and it wasn't clear to me
what the behaviour should be for the various scenarios.
The behaviour is briefly documented in
https://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/AMQP_1.0.
A link on which messages are to be sent out from the broker, i.e. a
subscription from the clients perspective, can have the 'shared'
capability requested on it. If two such subscription requests are made
with the same link name, then messages for that subscription are
distributed between those links, rather than each receiving their own copy.
This came out of discussions on the user list:
http://qpid.2158936.n2.nabble.com/c-Status-of-the-AMQP-1-0-work-td7591137.html#a7594079
Ideally we'd like to conform to a common standard. Does anyone know
if there were any plans to register this (or another) shared
subscription behaviour with the working group as an extension?
JMS 2.0 will need something similar (though more complicated), so at
some point the JMS binding doc will presumably suggest something.
Re: AMQP 1.0 and Shared Subscriptions
- Original Message - From: Ted Ross tr...@redhat.com To: proton@qpid.apache.org Sent: Wednesday, May 6, 2015 10:25:32 AM Subject: Re: AMQP 1.0 and Shared Subscriptions On 05/06/2015 10:05 AM, Ken Giusti wrote: Bump this up. I'm essentially in the same boat. The olso.messaging library in Openstack has introduced a feature called 'consumer pools' which is essentially the same pattern: a set of subscribers to a topic can belong to a pool. Each distinct pool of subscribers get a copy of the message sent to the topic, and only one subscriber from the pool claims the message. This pattern is available in the JMS 2.0 API via the createdSharedConsumer call [1] I'm particularly interested in how such a feature could be implemented in a qpid-dispatch based federation of brokers. I suspect that all subscribers to a given share would have to belong on the same broker - if shares existed on multiple brokers then each 'sub-share' would get a copy of the same message in violation of the pattern. A half-baked idea: What if we implemented a shared-target in Dispatch? It's like a waypoint except it doesn't have any external component and has no connections or links associated with it. You would configure a shared-target to receive from topic address T and send to pool address P. The semantics for T are multicast (i.e. each pool receiving from T will get one copy of every message from T) and the semantics for P are anycast (i.e. messages to P will be sent to exactly one of the consumers of P). Would that solve your problem? Openstack's particular case may be better solved on the broker due to their store-and-forward requirements on notification messages. oslo.messaging expects notifications get queued if there are no current consumers. In oslo.messaging the notification topic actually acts like a queue - only one subscriber consumes the message, and messages are balanced across all subscribers. A shared pool essentially adds another queue, with each pool getting its own copy of the message. A topic may have a mix of shared and non-shared subscribers. So the non-pooled subscribers in effect share a default 'anonymous' queue. I don't -think- it's required to queue messages for a shared pool, since a pool doesn't exist until there are actually subscribers to it. So maybe we implement a hybrid solution similar to what you describe: queue a copy of each message to the default queue using a queue route, and anycast a copy to each known pool consumer. That could buy some extra scalability. [1] http://www.oracle.com/technetwork/articles/java/jms2messaging-1954190.html - Original Message - From: Dominic Evans dominic.ev...@uk.ibm.com To: proton@qpid.apache.org Sent: Wednesday, May 6, 2015 7:41:59 AM Subject: AMQP 1.0 and Shared Subscriptions When we were implementing the MQ Light broker, we wanted to be able to support sharing of subscriptions across a group of clients - in particular for the worker offload scenario. At the time we were unable to find guidance in the specification or extensions thereof, so we went ahead with encoding this into the terminus addresses of an attach. Using a `private:` prefix to denote an exclusive subscription, and `share:sharename` to indicate a shared one. i.e., - @attach(18) [name=share:sharename:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] vs - @attach(18) [name=private:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=private:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=private:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] The other day I happened to notice that the qpid-cpp broker also supports a similar functionality since https://issues.apache.org/jira/browse/QPID-4917 / https://github.com/apache/qpid/commit/d4dafd3 - and it does this by supporting a capability of 'shared' on the exchange, and expecting a client @attach to request this capability, whereupon the link-name is used as-is for the name of the share. I wasn't able to find this behaviour documented any further than that, and it wasn't clear to me what the behaviour should be for the various scenarios. Ideally we'd like to conform to a common standard. Does anyone know if there were any plans to register this (or another) shared subscription behaviour with the working group as an extension? -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth
Re: AMQP 1.0 and Shared Subscriptions
On 6 May 2015 at 15:54, Gordon Sim g...@redhat.com wrote: Moving to the user list as this is a more general topic. On 05/06/2015 12:41 PM, Dominic Evans wrote: When we were implementing the MQ Light broker, we wanted to be able to support sharing of subscriptions across a group of clients - in particular for the worker offload scenario. At the time we were unable to find guidance in the specification or extensions thereof, so we went ahead with encoding this into the terminus addresses of an attach. Using a `private:` prefix to denote an exclusive subscription, and `share:sharename` to indicate a shared one. i.e., - @attach(18) [name=share:sharename:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=share:sharename:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] vs - @attach(18) [name=private:topicname, handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=0, source=@source(40) [address=private:topicname, durable=0, timeout=0, dynamic=false], target=@target(41) [address=private:topicname, durable=0, timeout=0, dynamic=false], initial-delivery-count=0] The other day I happened to notice that the qpid-cpp broker also supports a similar functionality since https://issues.apache.org/jira/browse/QPID-4917 / https://github.com/apache/qpid/commit/d4dafd3 - and it does this by supporting a capability of 'shared' on the exchange, and expecting a client @attach to request this capability, whereupon the link-name is used as-is for the name of the share. I wasn't able to find this behaviour documented any further than that, and it wasn't clear to me what the behaviour should be for the various scenarios. The behaviour is briefly documented in https://svn.apache.org/repos/asf/qpid/trunk/qpid/cpp/AMQP_1.0. A link on which messages are to be sent out from the broker, i.e. a subscription from the clients perspective, can have the 'shared' capability requested on it. If two such subscription requests are made with the same link name, then messages for that subscription are distributed between those links, rather than each receiving their own copy. This came out of discussions on the user list: http://qpid.2158936.n2.nabble.com/c-Status-of-the-AMQP-1-0-work-td7591137.html#a7594079 Ideally we'd like to conform to a common standard. Does anyone know if there were any plans to register this (or another) shared subscription behaviour with the working group as an extension? JMS 2.0 will need something similar (though more complicated), so at some point the JMS binding doc will presumably suggest something. As Gordon says, when we get to JMS 2.0 the AMQP JMS Mapping doc would need something in this area, since JMS 2.0 has shared durable and non durable subscriptions. We are initially doing a 1.1 mapping, but did spend some time thinking on this to see how it might work in with e.g the old non-shared durable subscriptions. We progressed an idea around an extended use the 'shared' capability. The main issue with using just the capability and link name is that links have to be unique (in a given direction) based on the conainer-id-1 + link-name + container-id-2 triplet, so for a particular 'broker container' that would limit you to having a single subscriber per set of connections using the same container id, which partly defeats the intent. As a result, we also ended up looking at ways to encode the subscription name into the address. That idea also came up independently because it was felt it would be better conceptually to have different address strings for different shared subscriptions on the same topic, given they would essentially be different [psudeo-] nodes. We eventually parked the idea and decided to revisit it when we actually get to JMS 2.0, and quite possibly look at using AMQP Management to do it instead, because things got horrendously complicated when adapting things further to enable meeting numerous additional restrictions needed for JMS subscription semantics. For those wonderring, some the issues for shared subs with JMS 2.0 included: the requirements for unsubscribing (where you dont know the topic), combined namespaces for shared and non shared durable subscriptions that must be distinguishable, sharing across different connections that might dont a clientid and so might have the same special containerid (one possibility considered for handling no-clientid), and creating new subscribers for a subscription with different topic+selector details which must fail in some situations but succeed in others. The list goes on. Robbie
