Hi Gordon,
Sorry I've been a little while getting back to you on this,
unfortunately I missed this response in my in-box.
Some of this is making me a little nervous in particular the response
around my headers Address String, but I might me misunderstanding e.g.
"That would now be handled in two steps. The first is a configuration
step whereby a 'topic' is created referencing amq.match and specifying
the desired 'policy' for subscription queues for that topic.
E.g. qpidt create topic amq.match.default exchange=amq.match
qpid.policy_type=ring qpid.max_size=500000000"
So this is using your new qpidt utility (what does qpidt stand for BTW?)
to create a "topic" called amq.match.default referencing amq.match with
the specified policy - correct? I'm not entirely sure what a "topic"
really relates to in the AMQP 1.0 sense but that's probably a separate
question....
"Then you would create your receiver using an address that referenced
the topic created, and specifying the filter instead of the x-bindings.
E.g.
amq.match.default; {link:{filter:{value:x-match:all,data-service:
amqp-delivery, item-owner:
fadams},name:data1,descriptor:"apache.org:legacy-amqp-headers-binding:map"}}}"
So this is using the "amq.match.default" topic in essence as a factory
and will create a subscription using what amounts to the same header
matches as my previous example using a 500K ring queue - correct?
So far so good, but in my example:
"testqueue; {create: receiver, node: {x-declare: {arguments:
{'qpid.policy_type': ring, 'qpid.max_size': 500000000}}, x-bindings:
[{exchange: 'amq.match', queue: 'testqueue', key: 'data1', arguments:
{x-match: all, data-service: amqp-delivery, item-owner: fadams}}]}}"
The fact that the queue had a particular name is significant! So in this
example the first consumer to create a destination with this Address
String causes a queue called "testqueue" to be created and binds it to
the headers exchange using the bindings specified in the x-bindings. But
(significantly for my use cases!) if that consumer were to close down
and restart it would simply reconnect to "testqueue" consuming any
messages that may have been left on it. Moreover if my consumer were to
note that the queue depth was rising too much I could scale out
additional instances which again would simply consume off "testqueue".
I guess what I'm saying is that the following use cases are significant
for me:
a) The ability to use *named* non-exclusive queue where consumers can
come and go and multiple consumers may be attached for scaling purposes.
b) My consumers operate in a "self-service" model, so a consumer creates
a destination via an Address String and that string may specify the
queue we wish to create.
c) There may be a non-trivial number of logical consumers, each of which
may use different queue names and each may or may not need to scale out
to multiple instances consuming of a given queue.
I'd be happy enough with the idea of "administratively" creating what
amounts to a single "factory Node" if all of my consumers were able to
use that to create their (different) queues in the sort of "self service
done via the Address String" model I allude to above, because all of my
queues tend to have the same parameters albeit with different names.
If I've just misinterpreted and you can suggest an Address String that
will do this sort of thing then fantastic, but I've got something of a
bad feeling that your example is much more akin to the "traditional"
topic subscription where consumers are logically consuming data off a
topic and the queue is incidental.
In your example Address String I expect that if I were to stand up
multiple consumers using that Address they'd each receive a copy of any
given message matching the said headers as opposed to receiving some of
the messages depending on how many consumer instances were consuming.
Thoughts?
Regards,
Frase
On 17/06/13 12:37, Gordon Sim wrote:
On 06/16/2013 10:46 AM, Fraser Adams wrote:
Hi Gordon,
I'm afraid that I've *still* not got around to messing with proton/AMQP
1.0 (sorry, I've been tied in knots with various other things) so I
won't be able to do the sort of review this really deserves.
Firstly though thanks for doing this, I think that being able to provide
this sort of configuration fixes a real limitation on queues
"automagically" created under the hood (does this work with
static/dynamic "exchange" routes too?).
This is AMQP 1.0 only. In 0-10 it is the client that actually create
the queue (and at that point the broker can't tell what the queue is
to be used for). In 1.0 the outgoing link is established from the
exchange (or topic with this modification in use) and the subscription
queue is under the full control of the broker which knows the context
in which it is to be used.
I notice that you've also done some work with respect to the headers
exchange x-binding stuff we discussed a little while back and similar
for xquery/xml exchange.
Yes, the client still doesn't accept the 'x-bindings' option, but
there is now a general purpose way of specifying 1.0 filters that
covers the simple cases (i.e. a single binding from subscription queue
to the exchange referenced as the node).
E.g.
my-xml-exchange;
{link:{filter:{name:foo,descriptor:"apache.org:xquery-filter:string",value:"declare
variable $colour external; $colour = 'red'"}}}
my-headers-exchange;
{link:{filter:{name:bar,descriptor:"apache.org:legacy-amqp-headers-binding:map",value:{x-match:all,colour:red,
shape:square}}}}
As mentioned before it would in some cases be possible to
auto-translate from x-bindings to filter syntax if there is sufficient
benefit to justify the effort.
I should also modify the 0-10 client so that it also recognises this
form of filter, meaning that one address could be used for either
protocol.
All of that is fantastic, so thanks very much. I don't suppose that
you'd be able to post syntax examples for all of these to the user list
(don't know if you've been able to update things like the Programming in
Apache Qpid book?).
No, I haven't yet updated the book. I must do so.
My headers bindings tend to follow a pattern similar
to below:
"testqueue; {create: receiver, node: {x-declare: {arguments:
{'qpid.policy_type': ring, 'qpid.max_size': 500000000}}, x-bindings:
[{exchange: 'amq.match', queue: 'testqueue', key: 'data1', arguments:
{x-match: all, data-service: amqp-delivery, item-owner: fadams}}]}}"
That would now be handled in two steps. The first is a configuration
step whereby a 'topic' is created referencing amq.match and specifying
the desired 'policy' for subscription queues for that topic.
E.g. qpidt create topic amq.match.default exchange=amq.match
qpid.policy_type=ring qpid.max_size=500000000
Then you would create your receiver using an address that referenced
the topic created, and specifying the filter instead of the
x-bindings. E.g.
amq.match.default; {link:{filter:{value:x-match:all,data-service:
amqp-delivery, item-owner:
fadams},name:data1,descriptor:"apache.org:legacy-amqp-headers-binding:map"}}}
All subscriptions from this new topic would use the default queue
configuration specified for the topic.
It was really this approach - define subscription queue config via
configuration rather than directly through the address used by
receivers - that I wanted your thoughts on (more than on the actual
code). Would this be an acceptable/useful solution? (Note, you can
create different topics referencing the same exchange if you want to
specify different policies for different types of use case).
TBH it took a bit of trial and error to figure out the headers binding
syntax at all (sometimes still seems a bit of a black art) so perhaps
the most constructive review comment that I can come up with is that
plenty of example address strings and other config. for various use
cases (including the new shared topic subscription if it needs any
special syntax) would be *really* useful.
Thanks again for this - and sorry again I can't yet do it justice.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
