On Fri, Apr 24, 2015 at 3:43 PM, Kevin Burton <bur...@spinn3r.com> wrote:
> On Fri, Apr 24, 2015 at 2:27 PM, Tim Bain <tb...@alumni.duke.edu> wrote: > > > If every message has at least one consumer for which the consumer's > > selector matches the message, you'll eventually process every message. > > > > That’s what I thought too, but that doesn’t work. > > > > Consumers that have no messages matching their selector in the cursor > will > > be delayed until the messages in front of their next one get consumed, > but > > they'll do it eventually; I don't think you'd have a complete failure to > > process messages as Kevin described. (Or maybe I'm reading the wrong > thing > > into Kevin's description. Kevin, can you confirm that you're getting NO > > messages to ANY consumer on your queue?) > > > > > Every message should be consumed. > > I have basically for selectors: > > artemis_priority > 4 > artemis_priority = 4 > artemis_priority < 4 > artemis_priority is null > > The last one was a fall through to consume messages once we first added the > artemis_priority header. > > (artemis is the name of our internal codebase) > > conceptually, I thought this might be an issue, which is why I designed the > selectors so that I don’t have to pre-read much. > > I also confirm that once I remove the selector, I immediately start > receiving messages that should match the above priorities. > > And if I change maxPageSize, it DOES work now … so eureka. That’s a big > step. > > Kevin, your screenshots didn't come through in my email client (Gmail) nor > > on the Nabble page; can you resend so we can see what you're seeing in > JMX? > > > > http://imgur.com/a/2myja What are the two screenshots; with and without the selector? If that's right, then clearly zero messages are matching the selector (which means this isn't a delivery problem, it's a selector problem), so hopefully pulling that thread will lead to a solution. > > Also, given the way cursors work, implementing priority using selectors > is > > never going to work. At best you'll only be able to prioritize among the > > first N messages in the store at any point in time (with N = the number > of > > messages that will fit into the cursor), which will eventually result in > > you having only N lowest-priority messages so you'll process the > > low-priority messages while your high-priority consumers sit unable to > > reach the high-priority messages deeper in the store. If you want to use > > selectors to implement priority, you're going to have to implement the > > enhancements to cursors that Jon and I were talking about on Wednesday. > > > > URL to this? Is this because with maxPageSize I’m getting partial > priority? The source of that info is mainly people like Art (especially) explaining cursor (mis)behavior to other people on the mailing list, but you can derive the info out of http://activemq.apache.org/message-cursors.html. And yes, it's because selectors only apply against messages in the cursor, not in the rest of the store, so maxPageSize limits how many messages your selectors can be applied to. If you consume the different priorities at roughly equivalent rates and if they're distributed well across your message stream, it shouldn't matter, but if you've got tons of high-priority consumers (or large batches of low-priority ones) you might end up with a situation where either there aren't any high-priority ones in the cursor (based on how they arrived) or where the many high-priority consumers cherry-pick out all the high-priority messages and all that's left is the low-priority ones. > If I’m already an in-memory store, is there any problem just setting > maxPageSize to a HUGE number? That's not code I've looked at, but I'd double-check the code to make sure it's not going to do anything crazy like pre-allocate a huge array/ArrayList or anything like that... > Kevin > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> >
