On 10 April 2013 13:38, Fraser Adams <[email protected]> wrote:
> Hi Gordon, > Comments inline below. > > Frase > > > On 09/04/13 19:07, Gordon Sim wrote: > >> On 6 April 2013 10:40, Fraser Adams <[email protected]**> >> wrote: >> >>> >>> [.. snip ..] > > Re "using a selector " I have to say that I'm definitely a big fan of > message selectors. It's worth pointing a few things out though. I suspect > that had message selectors been available when I started out with my system > I might well have used the topic exchange (with a named queue!) plus > message selectors. I've used JMS a lot in the past and that's at pattern > that worked well and was very flexible, however as I said earlier the topic > exchange on its own didn't cut the mustard for my needs so my producers > describe their data purely by means of properties/headers. > > I guess you know this already but message selectors tend to behave as > filters, so as I understand it messages would pass from exchange to queue > and the selector would apply a filter to what gets passed off the queue to > a consumer. I guess that to my way of thinking selectors provide most > benefit in the realm of "fine grained" filtering in that they allow quite > sophisticated boolean expressions to be applied. > > Now in my scenario quite clearly message selectors will be a boon and > offer some really useful fine grained control (I've wanted to be able to do > "not" for ages - you might just about recall one of my very early posts on > the headers exchange asking about a not operator). However I think that > message selectors are best applied *in addition to* a subscription, without > that loads of messages get delivered onto the queue which the consumer > isn't interested in, which a) fills up the queue unnecessarily and b) > overworks the message selector logic (which I suspect is more expensive > than the exchange selection logic). In my case in particular producers > deliver quite a diverse range of data at very high rate so filling up > consumer queues unnecessarily would be pretty bad news. > > So in précis while I really like message selectors I'd want them *in > addition to* headers bindings and good granularity of queue control (I > don't ask for much). > So, just for info, the Java Broker currently allows for selectors to be applied either at the consumer, or (for the Topic Exchange only, but this should be made universal) on the binding ... that is the selector becomes a further refinement of the binding preventing ingress of non-matching messages into the queue. > > Another point about selectors. As I say I'm excited about them, but they > are a new feature in the C++ broker (should be in 0.22 is that correct?). > Andrew has done some great work so far, but he has very much said that the > initial priority is on features and not interoperability. I'm fine with > that initially, but you will undoubtedly have gathered from many of my > posts that I'm really keen on maximum interoperability and cohesion. I'd > really like to see the C++ message selectors ultimately become completely > interoperable with JMS Message selectors. I've got no problem with an > "x-selector" heading in the Address string but I'm really meaning at an API > level too. I'd like to see JMS message selectors from JMS Clients propagate > to the C++ and Java brokers and I'd like to see qpid::messaging extended to > allow a JMS style message selector string to by passed to the > createReceiver() call on Session. Clearly it's early days for C++ broker > message selectors, but if that strategy wasn't part of the roadmap I'd be > pretty uncomfortable. > > +1 ... I think it's unfortunate that the current implementation doesn't seem to match the Java implementation (nor do I remember this choice being discussed on list). There is a definition of what a an AMQP 1.0 selector should look like and we should probably all try to converge on that, but in the meantime increasing commonality rather than disparity would be nice. > > The key then >> (if I understand the usecase) would be to ensure that the subscrption had >> the right lifecycle (i.e. you could reconnect to it without loss of >> messages while disconnected), >> > Well the lifecycle aspects are certainly part of it, hopefully my comments > above have explained the other aspects of the set of use cases that are > fairly critical to me. > > > > One thing that's not clear from your write up is whether the x-bindings >>> property you are talking about is in the messaging client, or whether it >>> is >>> the underlying x-bindings property that gets passed to the broker. I'm >>> guessing the latter which would also affect AddressStrings created by JMS >>> consumers and via QMF? (Although I use the C++ broker most of my >>> consumers >>> use JMS). >>> >>> It is in the messaging client. The x-bindings (and x-declare, >> x-subscribe), >> map directly to 0-10 commands. There is no analogous mechanism for >> *generic* binding operations in 1.0 - i.e. the ability to bind almost >> anything to anything, regardless of the node you are sending to or >> receiving from. However when you 'subscribe' to an exchange node, the >> broker can infer bindings from the link attachment, generally through the >> specification of a filter. In your case it sounds like that might be the >> applicable pattern over 1.0 and given the matching is based on examination >> of headers it could even be a selector. >> >> I guess I have to admit that I don't understand what you are saying > entirely, to be honest one common theme that seems to be building in my > mind is that AMQP 1.0 seems somewhat to be a bit of a retrograde step over > AMQP 0.10. I'm sure my current lack of understanding is making my initial > assumption a tad unfair but would you be able to summarise the "compelling > features" that AMQP 1.0 offers over AMQP 0.10 (aside from AMQP 1.0 having > been ratified by OASIS - that's probably a good thing, but how much "design > by committee" goes on at OASIS? is it closer to IETF or closer to ITU-T?) > > So - if you are a user of Qpid then there should not be any "retrograde" step when moving to using AMQP 1.0 we should be exposing all the existing functionality in some way. However what 1.0 does have is the ability for you to move to other vendors products presuming that you have not used vendor specific features. The issue with AMQP 0-x was that unless you were building a new broker brom scratch, your underlying broker model would not be the AMQP 0-x one... as such there is only Qpid implementing AMQP 0-10. With AMQP 1.0 we already see multiple implementations in Qpid, ActiveMQ, SwiftMQ, and Microsoft ServiceBus with more to come. However these other products won't be offering the "exchange" / "queue" AMQP 0-x model... but will instead be working with addresses you can send to, and receive from (with selectors). > So knee jerk comments aside :-) Re "It is in the messaging client. " does > that statement imply then that it remains possible to create the queues and > bindings to the headers exchange via QMF "create" calls to the Broker? If > so then that's something positive, but I'm then confused by the statement > "There is no analogous mechanism for *generic* binding operations in 1.0 - > i.e. the ability to bind almost anything to anything, regardless of the > node you are sending to or receiving from ", if it is possible to do this > by QMF "create" surely that remains a mechanism for generic binding. Sorry > I'm sure I'm being dumb, I really need to find the time to read up on the > AMQP 1.0 spec. > > The management of the broker entities (and their attributes and operations) is outside of the core specification of AMQP 1.0. The mechanism by which queues/topics are created in ActiveMQ, SwiftMQ or ServiceBus may be completely different from how queues/topics are created in Qpid. Thus there is no "generic" mechanism. If you have already made a choice to go with a particular vendor then mechanisms for creating entities within the broker will likely be available... and the AMQP Management specification which we are currently working on will define a generic mechanism for doing this. However this generic mechanism does not guarantee complete portability of code which attempts to modify broker configuration. The entity types and attributes within one implementation will not be guarantee to be the same as those in another. it is very unlikely that ActiveMQ or ServiceBus would be implementing the "HeadersExchange" in quite that way - though equivalent functionality would likely be available. > I guess in simple terms what I'm asking is how on earth do you set up a > subscription to a headers exchange in AMQP 1.0 along with the ability to > control the "buffering" between producer and consumer - pretty much what my > Address example above is doing. > > > So I guess the answer to the question depends on whether your question is " I guess in simple terms what I'm asking is how on earth do you set up a subscription to a headers exchange in AMQP 1.0 *in Qpid* ?" or not. Without specifying that you are using Qpid then there may be no answer because there may be no headers exchange. If you are using qpid then we should be able to give you an answer. > > One thing I should make clear is that 0-10 is not going away any time >> soon. >> The intention is not to force anyone onto 1.0, but simply to make it >> available (and indeed I hope eventually the default for new developments). >> > Yeah I realise that Gordon, perhaps *panic* is too strong a term for my > feelings :-) There are clearly issues though, now that AMQP 1.0 has been > ratified as you say it does certainly seem to be something we ultimately > need to migrate to even if not immediately. I've mentioned to you and Rob > before that I'd really like to see some published roadmaps for Qpid. So > "not going away any time soon" is one thing but a statement that we intend > to support AMQP 0.10 indefinitely, or until December 2015 (random date!!) > or whatever would give people confidence or conversely an opportunity to > plan their own architectural changes well in advance. > > I think one good thing about AMQP 1.0 being a ratified standard is that it > opens up opportunities for integration. A couple of our consumers actually > use ActiveMQ internally (and currently use a Camel based bridge IIRC), it'd > be much more elegant to federate directly from qpidd to ActiveMQ - I'm > assuming that having a ratified standard would make that sort of thing more > of a possibility ultimately. > > As an aside Qpid remains at a 0.x version. I guess that I had sort of > assumed that there would be an intention to uplift to Qpid 1.x when AMQP > 1.0 support had stabilised and was proved to be as reliable as the AMQP > 0.10 support, but TBH I've not seen any traffic at all relating to the > version? Any thoughts? I have to be honest I'd bet that the 0.x version > structure puts off a whole bunch of people who would really benefit from > using Qpid. To get on my "branding" hobby horse again we need to consider > just quite how conservative many large Enterprises actually are, it's hard > enough to get some of them to consider Open Source, but version 0.x to many > implies "unreliable and unstable" which Qpid is anything but. I think we're > doing ourselves a great dis-service if we don't put some consideration into > what it would take to become version 1.0 > > > Agreed - there are a few other things I would like to consider before moving to a 1.0 - in particular whether we actually want to continue to attempt to release everything as a monolithic release. -- Rob
