[
https://issues.apache.org/jira/browse/QPIDJMS-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673647#comment-15673647
]
Robbie Gemmell edited comment on QPIDJMS-220 at 1/12/17 2:40 PM:
-----------------------------------------------------------------
The following details an idea around a mechanism to support shared
subscriptions, and how that can be used in the JMS client.
h4. Short Version:
A 'SHARED-SUBS' connection or link capability is used to establish support for
the shared subscriptions mechanism.
A named \[shared\] subscription is identified using the link name.
The subscription identity can be scoped to the AMQP container, i.e. it is
unique within the scope of connections using the same container-id, or it can
be global, in which case the subscription identity is independent of the
container-id and can be used by receivers on any connection with any
container-id. A global subscription identity is indicated by adding the symbol
'global' to the Source capabilities.
A subscription may be shared by more than one consuming link, in which case the
messages for the subscription are distributed between all the consuming links.
To indicate that the link is for a shared subscription the symbol 'shared' is
added to Source capabilities.
AMQP requires that link names are unique in a given direction between two
containers, identified by the container-id specified in the Open frame. In
order to allow multiple links consuming from the same shared subscription on
connection(s) with the same container-id a link naming scheme is proposed such
that only the part of the link name up to (but excluding) a '|' character is
used for establishing the name of the subscription the link refers to. E.g. the
link names "mysub|foo" and "mysub|bar" would both be interpreted as referencing
a subscription named 'mysub' (which could then be durable or non-durable, and
global or container scoped), but as the two names are different they could be
used on the same connection or connections with the same container-id. Clients
would then utilise this where necessary to ensure unique link names are
utilised. The details after the "|" have no semantics associated with them, and
are only used to distinguish one link name from another while referencing the
same subscription name.
==============
h4. Longer version:
h6. Intro/Background:
JMS 1.1 has volatile and durable single-consumer topic subscriptions. Durable
subscriptions have a name, and these names are scoped within a ClientID, with a
single Connection being able to use a given ClientID at a time, and a ClientID
being mandatory to use a durable subscription.
JMS 2.0 added support for shared volatile and durable subscriptions, such that
for a given subscription name multiple consumers can consume from the same
subscription to spread processing load. It also makes ClientID optional for
these shared subscriptions, with different behaviour given depending on whether
a ClientID is set or not. If a ClientID is set, this means that only
subscribers on the same Connection can share the subscription. If a ClientID is
not set then subscribers from any Connection without a ClientID can share the
subscription. Non-shared durable subscriptions require a ClientID as in JMS 1.1
before. Shared durable and non-shared durable subscriptions use a single a name
space when a ClientID is set and can't co-exist.
The above essentially gives 5 separate subscription spaces:
Durable (shared + non-shared) with ClientID
Shared Volatile with ClientID
Shared Durable without ClientID
Shared Volatile without ClientID
Non-shared Volatile (regardless of ClientID status)
h6. Existing non-shared subscriber behaviour (used for JMS 1.1):
For single-consumer durable topic subscriptions, the client attaches a
receiving link, using the subscription name as the link name, and with a Source
having the address of the topic, a terminus durability of 1 or 2, a terminus
expiry of never, and filters to reflect selector/no-local. The connections
container-id contains the ClientID (which is required for non-shared durable
subscriptions) and as links are scoped to container-id this satisfies the
ClientID-scoping required for such subscriptions names. Closing (rather than
just detaching) the link represents end/unsubscribe of the subscription, and
this is utilised by Session#unsubscribe by attaching a receiving link using the
subscription name, with a null Source, requiring the server to perform a lookup
for an existing subscription using this link name. If none is found then the
link is refused and the client throws an exception, otherwise it closes the
opened link to provoke unsubscribe.
Link Name Examples:
{noformat}
With a ClientID: mySubscriptionName
Without ClientID: N/A, must have ClientID to use.
{noformat}
For single-consumer volatile topic subscriptions, the client attaches a
receiving link, using a generated link name, and with a Source having the
address of the topic, a terminus durability of 0, a terminus expiry of
link-detach, and filters to reflect selector/no-local.
Link Name Examples:
{noformat}
With or without a ClientID: <generated>
{noformat}
h6. New shared subscriber behaviour:
We need a mechanism to support the additional shared (and existing non-shared)
subscription behaviours, being able to convey:
\- Whether the mechanism is requested / supported / in-use.
\- A subscription name (accounting for required name spacing).
\- The address of the topic being subscribed to.
\- Whether the subscription is durable (and implicitly also how/when it could
be/is removed)
\- Whether it is shared or not.
\- Any selector or no-local (N/A for shared sub) filters.
\- We need to be able to signal removal of durable subscriptions given only the
subscription name, as that is all Session#unsubscribe gives us.
\- Implicitly or explicitly how to handle change of address/filters for a given
subscription.
h6. Inspecting support for shared subs
Since JMS explicitly offers the shared subscriptions via API, and it is
actually going to be supported via a layered mechanism, we need a mechanism to
detect \[lack of\] support and avoid confusion where specific servers don't
actually support it.
The proposal is to use a "SHARED-SUBS" connection capability or link capability
for this. If a server offers the connection capability, it is indicating that
any topic address can be used for shared subscriptions. A simple broker might
do this for example. On the other hand, a given server might not want/be able
to support this for any address but rather only specific addresses. Imagine
that Qpid Dispatch Router is being used to accept connections and facilitate
link routing to back end brokers, but additionally also facilitate direct
client to client messaging through the router mesh. In this scenario, some
addresses may support this behaviour while others might not, and so the router
offering the connection capability would be inappropriate.
If the connection capability is not offered by a server, the client can instead
use a link capability for detecting support. If the opened link does not offer
support for the capability when requested, the client can interpret this as a
lack of support and throw an exception accordingly.
h6. Shared Volatile Subscription
Starting from the non-shared volatile scenario, we additionally need to convey
that it is a shared sub, give the subscription name, and whether to share the
subscription within a single container or across multiple containers (i.e the
ClientID set vs no ClientID scenario).
We need to indicate that we want to share the subscription between multiple
subscribers by adding a "shared" capability to the Source. Similarly to the old
durable subscription case, we make the link name contain the subscription name.
AMQP requires that link names are unique in a given direction between two named
containers, however we also desire sharing on a single connection (in JMS, this
is actually required to share at all when a ClientID is set). To facilitate
this, we need a scheme by which such links can be disambiguated while still
retaining the ability to identify the same subscription name. To do this, we
propose to consider only the part of the link name up to (but excluding) a '|'
character for establishing the identity of the subscription the link refers to,
with the remainder being used only to disambiguate the different links from
each other. E.g. the link names "mysub|foo" and "mysub|bar" would both be
interpreted as referencing subscription name 'mysub' (which could be durable or
non-durable, and global or container scoped) but but as the two names are
different they could be used on the same connection. The detail after the "|"
has no semantics associated with it, and is only used to distinguish one links
name from another using the same subscription name, for example by using a
counter to index names for a subscription, and qualifiers to aid operators.
For the shared volatile subscription, the link terminus expiry is 'link-detach'
and there is no behavioural difference associated with whether the link is
detached with close=true or false, the subscription is considered ended when
the last subscriber detaches.
Link Name Examples:
{noformat}
With a ClientID: mySubscriptionName|volatile1 (then 2,3,etc)
Without ClientID: mySubscriptionName|global-volatile1 (then 2,3,etc)
{noformat}
h6. Shared Durable Subscription
Starting from the non-shared durable scenario. We additionally need to convey
that it is a shared sub, and whether to share the subscription within a single
container or across multiple containers (i.e the ClientID set vs no ClientID
scenario).
We again use the "shared" and "global" source capabilities as outlined for the
volatile subscription. Since shared and non-shared durable subscriptions can't
coexist with the same name, they use the same initial link name. Subsequent
shared subscribers for a given shared subscription then distinguish their link
names with a suffix using the mechanism described above. When a ClientID is not
set, the link is qualified with a suffix indicating it is a global subscription
link.
For the shared durable subscription, the link terminus expiry is 'never' and
the terminus is durable. Detaching the link leaves the subscription in place.
For the purposes of ending the subscription, closing the link is used to signal
intent to end the subscription (as in the non-shared case). If there are no
other active consumers this succeeds, but if there are other active consumer
links a detach with error is returned to the client.
{noformat}
With a ClientID: mySubscriptionName
With a ClientID: mySubscriptionName|2 (then 3,etc)
Without ClientID: mySubscriptionName|global
Without ClientID: mySubscriptionName|global2 (then 3,etc)
{noformat}
h6. Unsubscribe
Closing (detach with closed=true) a link ends the subscription, except if in
use by other subscribers (see server enforcement details below). For non-JMS
clients this can be any existing link. For JMS clients, a specific new link
must be attached knowing only the name of the subscription, as unsubscribe is
an explicit operation of the session.
During unsubscribe, if a ClientID is set on the connection, a link named with
the subscription name would be used to perform a 'null source lookup' for the
subscription, as it is already for the existing non-shared durable
subscriptions (see earlier for behaviour outline). If a ClientID is not set,
the "|global" suffix will be added as shown previously, e.g:
{noformat}
With a ClientID: mySubscriptionName
Without ClientID: mySubscriptionName|global
{noformat}
Additionally, while using connections that dont have a ClientID set, i.e using
global shared susbcriptions, the link will have "shared" and "global" desired
capabilities added as hints to the server that this is an attempt to attach to
a 'global' shared subscription of the appropriate name derived from the link,
aiding the server should no link with this name be known by it for the
particular client container-id currently in use.
h6. Server handling:
Servers will need to handle/provide the following behaviours:
\- Durable and volatile subscribers are distinct, may use same subscription
names (but not *link* names) concurrently.
\- Container-scoped and 'global' subscribers being distinct, i.e may use same
subscription names (but not *link* names) concurrently.
\- Refusing a shared subscriber link using the same link name as for an
existing non-shared subscription (and vice versa).
\- Refusing a global subscriber link using the same link name as for an
existing non-global subscription (and vice versa).
\- Changing topic/selector and reinitializing an existing sub with no active
subscribers
\- Refusing link trying to change topic/selector for existing sub with active
subscribers
\- Refusing to end an otherwise active subscription (e.g unsubscribe from
another connection while there is no ClientID set)
was (Author: gemmellr):
The following details an idea around a mechanism to support shared
subscriptions, and how that can be used in the JMS client.
h4. Short Version:
A 'SHARED-SUBS' connection or link capability is used to establish support for
the shared subscriptions mechanism.
A named \[shared\] subscription is identified using the link name.
The subscription identity can be scoped to the AMQP container, i.e. it is
unique within the scope of connections using the same container-id, or it can
be global, in which case the subscription identity is independent of the
container-id and can be used by receivers on any connection with any
container-id. A global subscription identity is indicated by adding the symbol
'global' to the Source capabilities.
A subscription may be shared by more than one consuming link, in which case the
messages for the subscription are distributed between all the consuming links.
To indicate that the link is for a shared subscription the symbol 'shared' is
added to Source capabilities.
AMQP requires that link names are unique in a given direction between two
containers, identified by the container-id specified in the Open frame. In
order to allow multiple links consuming from the same shared subscription on
connection(s) with the same container-id a link naming scheme is proposed such
that only the part of the link name up to (but excluding) a '|' character is
used for establishing the name of the subscription the link refers to. E.g. the
link names "mysub|foo" and "mysub|bar" would both be interpreted as referencing
a subscription named 'mysub' (which could then be durable or non-durable, and
global or container scoped), but as the two names are different they could be
used on the same connection or connections with the same container-id. Clients
would then utilise this where necessary to ensure unique link names are
utilised. The details after the "|" have no semantics associated with them, and
are only used to distinguish one link name from another while referencing the
same subscription name.
==============
h4. Longer version:
h6. Intro/Background:
JMS 1.1 has volatile and durable single-consumer topic subscriptions. Durable
subscriptions have a name, and these names are scoped within a ClientID, with a
single Connection being able to use a given ClientID at a time, and a ClientID
being mandatory to use a durable subscription.
JMS 2.0 added support for shared volatile and durable subscriptions, such that
for a given subscription name multiple consumers can consume from the same
subscription to spread processing load. It also makes ClientID optional for
these shared subscriptions, with different behaviour given depending on whether
a ClientID is set or not. If a ClientID is set, this means that only
subscribers on the same Connection can share the subscription. If a ClientID is
not set then subscribers from any Connection without a ClientID can share the
subscription. Non-shared durable subscriptions require a ClientID as in JMS 1.1
before. Shared durable and non-shared durable subscriptions use a single a name
space when a ClientID is set and can't co-exist.
The above essentially gives 5 separate subscription spaces:
Durable (shared + non-shared) with ClientID
Shared Volatile with ClientID
Shared Durable without ClientID
Shared Volatile without ClientID
Non-shared Volatile (regardless of ClientID status)
h6. Existing non-shared subscriber behaviour (used for JMS 1.1):
For single-consumer durable topic subscriptions, the client attaches a
receiving link, using the subscription name as the link name, and with a Source
having the address of the topic, a terminus durability of 1 or 2, a terminus
expiry of never, and filters to reflect selector/no-local. The connections
container-id contains the ClientID (which is required for non-shared durable
subscriptions) and as links are scoped to container-id this satisfies the
ClientID-scoping required for such subscriptions names. Closing (rather than
just detaching) the link represents end/unsubscribe of the subscription, and
this is utilised by Session#unsubscribe by attaching a receiving link using the
subscription name, with a null Source, requiring the server to perform a lookup
for an existing subscription using this link name. If none is found then the
link is refused and the client throws an exception, otherwise it closes the
opened link to provoke unsubscribe.
Link Name Examples:
{noformat}
With a ClientID: mySubscriptionName
Without ClientID: N/A, must have ClientID to use.
{noformat}
For single-consumer volatile topic subscriptions, the client attaches a
receiving link, using a generated link name, and with a Source having the
address of the topic, a terminus durability of 0, a terminus expiry of
link-detach, and filters to reflect selector/no-local.
Link Name Examples:
{noformat}
With or without a ClientID: <generated>
{noformat}
h6. New shared subscriber behaviour:
We need a mechanism to support the additional shared (and existing non-shared)
subscription behaviours, being able to convey:
\- Whether the mechanism is requested / supported / in-use.
\- A subscription name (accounting for required name spacing).
\- The address of the topic being subscribed to.
\- Whether the subscription is durable (and implicitly also how/when it could
be/is removed)
\- Whether it is shared or not.
\- Any selector or no-local (N/A for shared sub) filters.
\- We need to be able to signal removal of durable subscriptions given only the
subscription name, as that is all Session#unsubscribe gives us.
\- Implicitly or explicitly how to handle change of address/filters for a given
subscription.
h6. Inspecting support for shared subs
Since JMS explicitly offers the shared subscriptions via API, and it is
actually going to be supported via a layered mechanism, we need a mechanism to
detect \[lack of\] support and avoid confusion where specific servers don't
actually support it.
The proposal is to use a "SHARED-SUBS" connection capability or link capability
for this. If a server offers the connection capability, it is indicating that
any topic address can be used for shared subscriptions. A simple broker might
do this for example. On the other hand, a given server might not want/be able
to support this for any address but rather only specific addresses. Imagine
that Qpid Dispatch Router is being used to accept connections and facilitate
link routing to back end brokers, but additionally also facilitate direct
client to client messaging through the router mesh. In this scenario, some
addresses may support this behaviour while others might not, and so the router
offering the connection capability would be inappropriate.
If the connection capability is not offered by a server, the client can instead
use a link capability for detecting support. If the opened link does not offer
support for the capability when requested, the client can interpret this as a
lack of support and throw an exception accordingly.
h6. Shared Volatile Subscription
Starting from the non-shared volatile scenario, we additionally need to convey
that it is a shared sub, give the subscription name, and whether to share the
subscription within a single container or across multiple containers (i.e the
ClientID set vs no ClientID scenario).
We need to indicate that we want to share the subscription between multiple
subscribers by adding a "shared" capability to the Source. Similarly to the old
durable subscription case, we make the link name contain the subscription name.
AMQP requires that link names are unique in a given direction between two named
containers, however we also desire sharing on a single connection (in JMS, this
is actually required to share at all when a ClientID is set). To facilitate
this, we need a scheme by which such links can be disambiguated while still
retaining the ability to identify the same subscription name. To do this, we
propose to consider only the part of the link name up to (but excluding) a '|'
character for establishing the identity of the subscription the link refers to,
with the remainder being used only to disambiguate the different links from
each other. E.g. the link names "mysub|foo" and "mysub|bar" would both be
interpreted as referencing subscription name 'mysub' (which could be durable or
non-durable, and global or container scoped) but but as the two names are
different they could be used on the same connection. The detail after the "|"
has no semantics associated with it, and is only used to distinguish one links
name from another using the same subscription name, for example by using a
counter to index names for a subscription, and qualifiers to aid operators.
For the shared volatile subscription, the link terminus expiry is 'link-detach'
and there is no behavioural difference associated with whether the link is
detached with close=true or false, the subscription is considered ended when
the last subscriber detaches.
Link Name Examples:
{noformat}
With a ClientID: mySubscriptionName|volatile1 (then 2,3,etc)
Without ClientID: mySubscriptionName|global-volatile1 (then 2,3,etc)
{noformat}
h6. Shared Durable Subscription
Starting from the non-shared durable scenario. We additionally need to convey
that it is a shared sub, and whether to share the subscription within a single
container or across multiple containers (i.e the ClientID set vs no ClientID
scenario).
We again use the "shared" and "global" source capabilities as outlined for the
volatile subscription. Since shared and non-shared durable subscriptions can't
coexist with the same name, they use the same initial link name. Subsequent
shared subscribers for a given shared subscription then distinguish their link
names with a suffix using the mechanism described above. When a ClientID is not
set, the link is qualified with a suffix indicating it is a global subscription
link.
For the shared durable subscription, the link terminus expiry is 'never' and
the terminus is durable. Detaching the link leaves the subscription in place.
For the purposes of ending the subscription, closing the link is used to signal
intent to end the subscription (as in the non-shared case). If there are no
other active consumers this succeeds, but if there are other active consumer
links a detach with error is returned to the client.
{noformat}
With a ClientID: mySubscriptionName
With a ClientID: mySubscriptionName|2 (then 3,etc)
Without ClientID: mySubscriptionName|global
Without ClientID: mySubscriptionName|global2 (then 3,etc)
{noformat}
h6. Unsubscribe
During unsubscribe, if a ClientID is set, a link named with the subscription
name would be used to perform a 'null source lookup' for the subscription, as
it is already for the existing non-shared durable subscriptions (see earlier
for behaviour outline). If a ClientID is not set, the "|global" suffix will be
added as shown previously, e.g:
{noformat}
With a ClientID: mySubscriptionName
Without ClientID: mySubscriptionName|global
{noformat}
Additionally, while using connections that dont have a ClientID set, i.e using
global shared susbcriptions, the link will have "shared" and "global" desired
capabilities added as hints to the server that this is an attempt to attach to
a 'global' shared subscription of the appropriate name derived from the link,
should no link of this name be known by the server for the particular
connection container-id currently in use.
h6. Server handling:
Servers will need to handle/provide the following behaviours:
\- Durable and volatile subscribers are distinct, may use same subscription
names (but not *link* names) concurrently.
\- Container-scoped and 'global' subscribers being distinct, i.e may use same
subscription names (but not *link* names) concurrently.
\- Refusing a shared subscriber link using the same link name as for an
existing non-shared subscription (and vice versa).
\- Refusing a global subscriber link using the same link name as for an
existing non-global subscription (and vice versa).
\- Changing topic/selector and reinitializing an existing sub with no active
subscribers
\- Refusing link trying to change topic/selector for existing sub with active
subscribers
\- Refusing to end an otherwise active subscription (e.g unsubscribe from
another connection while there is no ClientID set)
> add support for shared topic subscriptions
> ------------------------------------------
>
> Key: QPIDJMS-220
> URL: https://issues.apache.org/jira/browse/QPIDJMS-220
> Project: Qpid JMS
> Issue Type: Sub-task
> Components: qpid-jms-client
> Reporter: Robbie Gemmell
> Assignee: Robbie Gemmell
>
> JMS 2.0 adds support for shared volatile and durable topic subscriptions.
> Whilst at its simplest this logically reduces to consuming messages from a
> shared backing queue representing the subscription, there are however many
> details to consider and a mechanism is needed for conveying the required
> details/behaviours using AMQP 1.0. This JIRA serves to capture information
> regarding this, both as it applies to AMQP clients/servers in general, and
> specifically to the JMS client.
> Sub-task of QPIDJMS-207
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
