Timothy, last question, I promise. Prefetch = 1 means that at most 2 messages will be dispatched to a consumer, the in-flight message and the "prefetched" one? I ask you because setting to one resulted in four consumers with dispatched counter = 1, three consumers with dispatched counter = 2, and the other three consumers waiting for messages, as seen via MBean. On Mon Feb 02 2015 at 8:34:48 PM Timothy Bish <tabish...@gmail.com> wrote:
> On 02/02/2015 05:27 PM, Thiago Kronig wrote: > > First of all, thank you Timothy and Tim for your time. > > > > In fact, setting jms.prefetchPolicy.all=0 solves the problem, because > > "*Specifying > > a prefetch limit of zero means the consumer will poll for more messages, > > one at a time"*. > > > > Please, validate my understanding: the default prefetch size is 1000 > > messages for some queue costumer, which means that all my 10 consumers > were > > actually competing to dequeue 1000 messages from the broker. The broker, > in > > some runnings dispatched 2 messages to one and 8 messages to another. > > With competing consumers that have prefetch things will eventuall smooth > out to where the broker will round robin dispatch but only after they > are all online, prior to that the broker will dispatch based on the > arrival of the consumers so the first one likely got seven then the > second got one and then first got one...or other variations. > > You could set prefetch to one, but you can still have a slight imbalance > depending on how fast the messages get processed on each consumer. You > need to determine the optimal configuration based on your use case and > requirements. > > > > > On Mon Feb 02 2015 at 7:59:26 PM Timothy Bish <tabish...@gmail.com> > wrote: > > > >> On 02/02/2015 04:47 PM, Thiago Kronig wrote: > >>> I just tried the same approach using JMS consumers. No success. > >>> Source at: > >>> https://github.com/thiagokronig/activemq-camel- > >> test/blob/master/src/main/java/com/thiagokronig/activemqcameltest/ > >> ActiveMQJMSParallelTest.java > >>> On Mon Feb 02 2015 at 7:30:00 PM Thiago Kronig <thiagokro...@gmail.com > > > >>> wrote: > >> You need to take consumer prefetch values into account when dealing with > >> concurrent consumers. > >> http://activemq.apache.org/what-is-the-prefetch-limit-for.html > >> > >>>> Tim, you're right: 1 second is to small timeout to begin with. > >>>> But even if I set to 10 minutes, I get the same discouraging results: > 3 > >>>> consumers, sometimes 8 consumers processing different messages in > >> parallel. > >>>> Sometimes 10 consumers, as my test requires, consumes all the messages > >> in > >>>> parallel and exit before the `10 minute` timeout; but that is > >> *sometimes*. > >>>> > >>>> On Mon Feb 02 2015 at 7:12:58 PM Tim Bain <tb...@alumni.duke.edu> > >> wrote: > >>>>> Are you sure that your assumption that a Camel context can be > >> initialized > >>>>> enough to consume messages in all 10 threads within 1 second is a > valid > >>>>> one? In my experience, Camel contexts aren't quick to start, and it > >>>>> wouldn't surprise me if there was some asynchronous startup logic > that > >>>>> took > >>>>> place in parallel after control returns from you call to > >>>>> camelContext.addRoutes(). Can you point us to any documentation that > >>>>> validates that startup is supposed to work as quickly as your code > >>>>> assumes? If not, you could increase the timeouts to 2 seconds, and I > >>>>> would > >>>>> expect your test would start succeeding most/all of the time... > >>>>> > >>>>> On Mon, Feb 2, 2015 at 1:49 PM, Thiago Kronig < > thiagokro...@gmail.com> > >>>>> wrote: > >>>>> > >>>>>> Tim, other thing: I tried to Thread.sleep(5000); between the > producer > >>>>>> commit and the consumer initialization, but had no success. > >>>>>> > >>>>>> On Mon Feb 02 2015 at 6:45:19 PM Thiago Kronig < > >> thiagokro...@gmail.com> > >>>>>> wrote: > >>>>>> > >>>>>>> Tim, in may scenario I want to consume N messages in N consumers, > one > >>>>> for > >>>>>>> each. My consumers wait on a barrier only to show the parallelism, > >>>>> and to > >>>>>>> show that none of them processed two messages or more. Maybe my > Camel > >>>>>>> configuration is wrong, or there is something wrong with the broker > >>>>>>> configuration. > >>>>>>> > >>>>>>> > >>>>>>> On Mon Feb 02 2015 at 6:17:38 PM Tim Bain <tb...@alumni.duke.edu> > >>>>> wrote: > >>>>>>>> Why do your consumers all wait for the Nth message to be received > >>>>> before > >>>>>>>> they return to process another? Why don't you use an > AtomicInteger > >>>>> to > >>>>>>>> subtract one each time, and succeed if you hit 0 and fail if you > >>>>> haven't > >>>>>>>> hit 0 by the end of your timeout interval? Based on what you've > >>>>> shows > >>>>>> us > >>>>>>>> so far, this appears to be an artifact of your test code, not a > >>>>> problem > >>>>>>>> with the broker. > >>>>>>>> > >>>>>>>> On Mon, Feb 2, 2015 at 1:02 PM, Thiago Kronig < > >>>>> thiagokro...@gmail.com> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi list. > >>>>>>>>> > >>>>>>>>> I'm trying to concurrently consume 10 messages sent to an > embedded > >>>>>>>> ActiveMQ > >>>>>>>>> broker over the VM transport. Sometimes my code works, sometimes > it > >>>>>>>> hangs > >>>>>>>>> at the n-th message, randomly. > >>>>>>>>> > >>>>>>>>> If I change to a JBossMQ, my Camel client works. > >>>>>>>>> > >>>>>>>>> Can someone help me in setting my ActiveMQ broker? > >>>>>>>>> > >>>>>>>>> Source at: https://github.com/thiagokronig/activemq-camel-test > >>>>>>>>> > >>>>>>>>> Thanks in advance. > >>>>>>>>> > >> > >> -- > >> Tim Bish > >> Sr Software Engineer | RedHat Inc. > >> tim.b...@redhat.com | www.redhat.com > >> skype: tabish121 | twitter: @tabish121 > >> blog: http://timbish.blogspot.com/ > >> > >> > > > -- > Tim Bish > Sr Software Engineer | RedHat Inc. > tim.b...@redhat.com | www.redhat.com > skype: tabish121 | twitter: @tabish121 > blog: http://timbish.blogspot.com/ > >
