Hi Matt,
Comments inline.
On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich <[email protected]>
wrote:
Martyn-
I think you nailed it here-- well done =)
My notes in-line--
On 11/21/16 10:45 AM, Martyn Taylor wrote:
1. Ability to route messages to queues with the same address, but
different
routing semantics.
The proposal outlined in ARTEMIS-780 outlines a new model that
introduces
an address object at the configuration and management layer. In the
proposal it is not possible to create 2 addresses with different
routing
types. This causes a problem with existing clients (JMS, STOMP and for
compatability with other vendors).
Potential Modification: Addresses can have multiple routing type
“endpoints”, either “multicast” only, “anycast” only or both. The
example
below would be used to represent a JMS Topic called “foo”, with a
single
subscription queue and a JMS Queue called “foo”. N.B. The actual XML
is
just an example, there are multiple ways this could be represented
that we
can define later.
<*addresses*> <*address* *name**="foo"*> <*anycast*>
<*queues*> <*queue* *name**="**foo”* /> </*queues*>
</*anycast*> <*mulcast*> <*queues*>
<*queue* *name**="my.topic.subscription" */> </*queues*>
</*multicast*> </*address*></*addresses*>
I think this solves it. The crux of the issues (for me) boils down to
auto-creation of destinations across protocols. Having this show up in
the
configs would give developers and admins more information to
troubleshoot
the mixed address type+protocol scenario.
2. Sending to “multicast”, “anycast” or “all”
As mentioned earlier JMS (and other clients such as STOMP via
prefixing)
allow the producer to identify the type of end point it would like to
send
to.
If a JMS client creates a producer and passes in a topic with address
“foo”. Then only the queues associated with the “multicast” section of
the
address. A similar thing happens when the JMS producer sends to a
“queue”
messages should be distributed amongst the queues associated with the
“anycast” section of the address.
There may also be a case when a producer does not identify the
endpoint
type, and simply sends to “foo”. AMQP or MQTT may want to do this. In
this
scenario both should happen. All the queues under the multicast
section
get
a copy of the message, and one queue under the anycast section gets
the
message.
Modification: None Needed. Internal APIs would need to be updated to
allow
this functionality.
I think the "deliver to all" scenario should be fine. This seems
analogous
to a CompositeDestination in ActiveMQ 5.x. I'll map through some
scenarios
and report back any gotchas.
3. Support for prefixes to identify endpoint types
Many clients, ActiveMQ 5.x, STOMP and potential clients from alternate
vendors, identify the endpoint type (in producer and consumer) using a
prefix notation.
e.g. queue:///foo
Which would identify:
<*addresses*> <*address* *name**="foo"*> <*anycast*>
<*queues*> <*queue* *name**="my.foo.queue" */>
</*queues*> </*anycast*> </*address*></*addresses*>
Modifications Needed: None to the model. An additional parameter to
the
acceptors should be added to identify the prefix.
Just as a check point in the syntax+naming convention in your provided
example... would the name actually be:
<*queue* *name**="foo" .. vs "my.foo.queue" ?
The queue name can be anything. It's the address that is used by
consumer/producer. The protocol handler / broker will decided which
queue
to connect to.
4. Multiple endpoints are defined, but client does not specify
“endpoint
routing type” when consuming
Handling cases where consumers does not pass enough information in
their
address or via protocol specific mechanisms to identify an endpoint.
Let’s
say an AMQP client, requests to subscribe to the address “foo”, but
passes
no extra information. In the cases where there are only a single
endpoint
type defined, the consumer would associated with that endpoint type.
However, when both endpoint types are defined, the protocol handler
does
not know whether to associate this consumer with a queue under the
“anycast” section, or whether to create a new queue under the
“multicast”
section. e.g.
Consume: “foo”
<*addresses*>
<*address* *name**="foo"*> <*anycast*> <*queues*>
<*queue* *name**="**foo”* /> </*queues*>
</*anycast*> <*multicast*> <*queues*> <*queue*
*name**="my.topic.subscription" */> </*queues*>
</*multicast*> </*address*></*addresses*>
In this scenario, we can make the default configurable on the
protocol/acceptor. Possible options for this could be:
“multicast”: Defaults multicast
“anycast”: Defaults to anycast
“error”: Returns an error to the client
Alternatively each protocol handler could handle this in the most
sensible
way for that protocol. MQTT might default to “multicast”, STOMP
“anycast”,
and AMQP to “error”.
Yep, this works great. I think there are two flags on the acceptors..
one
for auto-create and one for default handling of name collision. The
defaults would most likely be the same.
Something along the lines of:
auto-create-default = "multicast | anycast"
no-prefix-default = "multicast | anycast | error"
5. Fully qualified address names
This feature allows a client to identify a particular address on a
specific
broker in a cluster. This could be achieved by the client using some
form
of address as:
queue:///host/broker/address/
Matt could you elaborate on the drivers behind this requirement
please.
I am of the opinion that this is out of the scope of the addressing
changes, and is more to do with redirecting in cluster scenarios. The
current model will support this address syntax if we want to use it in
the
future.
I agree that tackling the impl of this should be out-of-scope. My
recommendation is to consider it in addressing now, so we can hopefully
avoid any breakage down the road.
A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, etc) is
the
ability to fully address a destination using a format similar to this:
queue://brokerB/myQueue
The advantage of this is to allow for scaling of the number of
destinations
and allows for more dynamic broker networks to be created without
applications having to have connection information for all brokers in a
broker network. Think simple delivery+routing, and not horizontal
scaling.
It is very analogous to SMTP mail routing.
Producer behavior:
1. Client X connects to brokerA and sends it a message addressed:
queue://brokerB/myQueue
2. brokerA accepts the message on behalf of brokerB and handles all
acknowledgement and persistence accordingly
3. brokerA would then store the message in a "queue" for brokerB. Note:
All messages for brokerB are generally stored in one queue-- this is
how it
helps with destination scaling
Broker to broker behavior:
There are generally two scenarios: always-on or periodic-check
In "always-on"
1. brokerA looks for a brokerB in its list of cluster connections and
then
sends all messages for all queues for brokerB (or brokerB pulls all
messages, depending on cluster connection config)
In "periodic-check"
1. brokerB connects to brokerA (or vice-versa) on a given time interval
and then receives any messages that have arrived since last check
TL;DR;
It would be cool to consider remote broker delivery for messages while
refactoring the address handling code. This would bring Artemis inline
with
the rest of the commercial EMS brokers. The impact now, hopefully, is
minor
and just thinking about default prefixes.
Understood, from our conversations on IRC I can see why this might be
useful.
Thanks,
-Matt
