On 04/19/2013 11:33 AM, Fraser Adams wrote:
Hmm, I'm not sure. I guess that it depends exactly what you mean by a policy on the exchange.
What I meant was that in many cases it is something you want to enforce for all subscribers (since often a key goal is to protect all users of the topic from delinquents), rather than have subscribers choose their own settings.
I don't mean that the exchange as a module of code would actually be doing the limiting. Merely that e.g. you would set the desired characteristics at the topic level, and that would be automatically picked up by default for all subscribers. (This could still be implemented as a circular queue per subscriber, but that choice would be driven by configuration of the 'node' not the 'link' if that makes any sense).
I've seen some of the work that has been going on on the "rebind" thread and being able to redirect messages to an alternate consume when queue threshold exceed triggers would be useful to me in some scenarios so I'm keen to loot at that, but the current circular queue model seems to fit my scenario the best in most circumstances. Having the exchange care about how much backlog subscribers are allowed suggests in my mind some sort of discard when that limit is reached. For sure there are cases where I'd be happy with that, and protecting producers and/or the well behaving consumers is high on my list of things I like. The nice thing about circular queues however is that the most recent item always hits the queue and it's the oldest items that get blatted if the queue depth grows. That's something I really need!!
Right, and I'm not suggesting anything different.
Indeed If I had a "wish list" what I'd *really* like is a fixed depth stack, in other words a lifo rather than a fifo. As I mentioned previously I've got a hight rate, low latency system and perhaps I should have mentioned that the most recent items are "generally" the most interesting,
Once you have read a more recent item, are older items still of interest? Or does a more recent item essentially invalidate the older ones (e.g. some sort of last value queue)?
[...]
So to be honest my issue is I suspect more subtle than just "pure technical". When I started architecting my system I'd have given my eye teeth and quite a few other body parts to have been able to use message selectors. I'd used them loads and frankly the boolean capabilities would have stopped me from losing the last vestige of sanity I once had :-/ However they didn't exist and that's partly my slight dilemma. I have a non-trivial number of consumers and therefore a potential non-trivial integration issue. I feel pretty heartened that being able to apply selectors between exchange and queue is ultimately good news but I'd be "very sad" if I had to change all subscriptions in lock-step. If it's possible to have a mixed economy whereby I can have some consumers using AMQP 0.10 and current headers bindings and some using AMQP 1.0 and selectors then I can have a controlled migration.
Right, and that is absolutely a goal for me. You can mix 0-10 and 1.0 connections and have messages transferred between them through the broker. You shouldn't need to alter the 0-10 parts at all.
In an ideal world I'd like to be able to pump controlled data into my reference system and have a consumer with headers bindings and a consumer with the equivalent selector so that I could prove any binding changes systematically too. I guess that the point is that I've got a current operational system so having to change any of the key characteristics introduces mission risk so I'd be uncomfortable without options to mitigate this. Clearly if the "to be" offers increased flexibility and control then that would really help. I hope that this makes sense. Please don't take any of this as being intentionally negative, I'm aware of the need to evolve, but OTOH I've got a mission critical Qpid based system that I'm keen to protect.
Understood. The 0-10 support in the messaging API isn't affected. Where you do want to start using 1.0 that is entirely under your control and certainly doesn't force a system wide change of protocol, nor should it ofrce you to change the exchanges and queues you are already using.
Re "but the matching logic in the headers exchange is not particularly clever" yeah, I'd gathered that. I've not looked in detail but it looked like it was doing linear searches. It'd be nice to see some benchmarks comparing headers matches with equivalent selectors. heh heh <evil>given that there's almost certainly a selector equivalent to a headers match it aught to be possible to take an existing headers binding and auto translate it - fancy giving that a go as a configurable option on qpid::messaging</evil> :-D
That sort of translation is possible for certain specific cases (the one you describe being an example, assuming of course a sensible exchange and queue). The issue really is a generic mapping of the x-bindings to 1.0.
I'm not opposed to doing some of these things, but I think the priority is for a clean and fully functional 1.0 mapping first, with temporary, under-the-covers translations added later based on some guesstimate of 'bang for buck'.
As an aside any idea if thought has gone into selectors with respect to binary Vs UTF8 strings. You may recall some interoperability comedy in the past whereby C++ (qpid::messaging at any rate) tended to default to setting headers as binary strings whereas Java would use UTF8. I'd be *very keen* to know if thought has gone in to string interoperability, it's very easy to miss without testing in a heterogeneous environment.
It's a good question. I haven't done any explicit testing of selectors mixing binary and string types (Andrew?).
For 1.0, a VAR_STRING typed variant will be written as a 1.0 'string' (i.e. a sequence of unicode chars) unless the encoding has been explicitly set to 'binary'. (The 0-10 path is unchanged).
I suppose all of this is also suggesting that the XML exchange is likely to be on its way out. I've never really used it myself but I expect people who do will be faced with similar questions.
The XML exchange is slightly different in that it can also match on the body content (assuming it is XML). It has been used as a richer header-based routing mechanism though, and that would no longer be needed over 1.0 with the improved selector support.
The exchange itself is not going away though, neither is the headers exchange. All I'm saying is that the use of selectors will offer a simpler, richer approach.
[...]
I guess that I don't know enough about the syntax yet. I was thinking primarily around JMS Message Selectors. It'll be somewhat unfortunate and I suspect open up fair scope for confusion if AMQP Message Selectors have radically different syntax and capabilities compared with JMS Message Selectors. TBH it would be a "right pain". Ultimately the JMS client would have to support JMS Message Selectors to be JMS compliant so there may be an implicit mapping to AMQP Message Selectors if they happen to be different. I don't know enough about the detail but it's making me a little nervous, perhaps Rob knows?
The current registered extension is essentially JMS selector syntax but with the 'JMS' prefixed special properties replaced by AMQP equivalents. The reason for that is to allow one syntax that makes sense even if you aren't using JMS. It is then trivial for a JMS client to translate the selector presented by the application to its AMQP equivalent (just a mapping of certain property names).
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
