You can setup per consumer prefetch sizes. If you can send to a different queue based on how long each message will take to process (for example fastqueue and slowqueue) Then you can setup consumers on fastqueue with a large prefetch and consumer on slowqueue with a small prefetch.
On Mon, Nov 5, 2012 at 2:22 AM, sdonovan_uk <sdonovan...@yahoo.com> wrote: > We're trying to use ActiveMQ as the queue for a distributed set of > calculation servers (think many machines with 64-cores on-board). > > Imagine a single queue with N consumers. The only efficient way to scale > is > to use consumer prefetch. However. Each of our messages can take between > 3ms and minutes to process (there is a huge range). When testing with 3ms > per message (with prefetch of 0 or 1) it is not scaling linearly past ~32 > consumers. > > Is there anyway (either through configuration, programming, etc.) to make > the prefetch a variable/dynamic value? Ideally, I'd like to add hints to > the message. > > Any suggestions would be much appreciated! > > Sean > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Variable-dynamic-prefetch-size-tp4658760.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > -- ** *Hiram Chirino* *Engineering | Red Hat, Inc.* *hchir...@redhat.com <hchir...@redhat.com> | fusesource.com | redhat.com* *skype: hiramchirino | twitter: @hiramchirino<http://twitter.com/hiramchirino> * *blog: Hiram Chirino's Bit Mojo <http://hiramchirino.com/blog/>*
