Sounds like a good idea. I just pushed with maxPageSize=5000
Interesting that my backlogged queues seems to be locked up at about 5000
messages.
… I’m wonder if those are messages that aren’t matching selectors?
I could have a plan B / fall through that accepts messages without a
selector.. it would be a hack and would give me soft priority but this
would get me closer to my goal.
throttle-queue:discovery-task:pending:192.0.78.13
5,004 4
throttle-queue:discovery-task:pending:192.0.78.12
5,002 4
throttle-queue:discovery-task:pending:74.125.141.132
5,000 0
throttle-queue:feed-task:pending:74.125.141.132
4,997 0
throttle-queue:source-task:pending:69.171.237.20
4,062 0
On Fri, Apr 24, 2015 at 3:40 PM, Tim Bain <tb...@alumni.duke.edu> wrote:
> OK, so based on what you wrote, it sounds like the broker gets into a state
> where message selectors don't work properly, since the same selector
> matches all messages (except ones with a null priority; have you confirmed
> that both artemis_priority <= 9 and artemis_priority = null match their
> respective messages when you're in a good state, and that neither one gets
> any messages when you're in a bad state?).
>
> I'd start the broker with the debug port enabled, pull up the
> selector-evaluation code and set a breakpoint in it (I'd start with
> SelectorWorker.run()), wait till you get into the bad state, and then
> attach a debugger and step through the code to see why messages aren't
> getting handed off to your consumer with the selector.
>
> Tim
>
> On Fri, Apr 24, 2015 at 4:27 PM, Kevin Burton <bur...@spinn3r.com> wrote:
>
> > > >
> > > > http://imgur.com/a/2myja
> > >
> > >
> > > What are the two screenshots; with and without the selector? If that's
> > > right, then clearly zero messages are matching the selector (which
> means
> > > this isn't a delivery problem, it's a selector problem), so hopefully
> > > pulling that thread will lead to a solution.
> > >
> > >
> > Actually, maybe it’s just a but related to message selectors?
> >
> > If I remove the selector, I have it print the messages it receives. I
> then
> > confirm that they are ALL have the right headers. So from a mathematical
> > perspective, they should match.
> >
> > Here’s an example.
> >
> > QueueDumperTask: Got message: using config: Config{sync=true,
> > selector='artemis_priority <= 9'}
> >
> > QueueDumperTask:
> > ID:util0048.wdc.sl.spinn3r.com-38049-1429893012757-6:1:1:1:11179
> >
> > QueueDumperTask: artemis.link=http://entertainment-2-all.blogspot.com
> >
> > QueueDumperTask: artemis.throttleKey=74.125.141.132
> >
> > QueueDumperTask: artemis.pending.timestamp=2015-04-24T16:46:30Z
> >
> > QueueDumperTask: artemis.sequence=1429893990000012414
> >
> > QueueDumperTask:
> > artemis.pendingQueue=throttle-queue:discovery-task:pending:74.125.141.132
> >
> > QueueDumperTask: artemis_priority=0
> >
> > QueueDumperTask: artemis.throttleQueue=discovery-task
> >
> > QueueDumperTask: artemis.delay=0
> >
> > QueueDumperTask: JMSXDeliveryCount=3
> >
> > .. so it should match
> >
> > artemis_priority <= 9
> >
> > … so I think there are two issues here. I’m aware of the improper
> > scheduling issue do to maxPageSize… so I won’t get REAL priorities. I’ll
> > get soft priorities.
> >
> > But that’s not the issue that I’m seeing.. What I’m seeing is that the
> > ENTIRE thing locks up. Because at least ONE of my selectors should work.
> >
> >
> > > > If I’m already an in-memory store, is there any problem just setting
> > > > maxPageSize to a HUGE number?
> > >
> > >
> > > That's not code I've looked at, but I'd double-check the code to make
> > sure
> > > it's not going to do anything crazy like pre-allocate a huge
> > > array/ArrayList or anything like that...
> > >
> >
> > Yes. That’s what I was worried about.. I might trace it or just try it in
> > production and see what memory usage looks like.
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
> > <http://spinn3r.com>
> >
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
