The documentation you linked to sounds like Artemis allows rate-limiting consumers individually but not rate-limiting an entire pool of consumers. I was under the impression that Bilgin had more than one consumer and wanted the ability to limit them as a group, but it wasn't entirely clear so maybe I read the wrong things between the lines.
Tim On Feb 2, 2016 6:59 AM, "Justin Bertram" <[email protected]> wrote: > For what it's worth, I believe the ActiveMQ Artemis broker (a sub-project > of ActiveMQ) has the feature you're looking for. See more details in the > documentation [1]. > > > Justin > > [1] http://activemq.apache.org/artemis/docs/1.2.0/flow-control.html > > ----- Original Message ----- > From: "Bilgin Ibryam" <[email protected]> > To: [email protected] > Sent: Tuesday, February 2, 2016 5:54:28 AM > Subject: Re: Consumer throttling > > Thanks all for your inputs. It seems there is no solution for my > question in ActiveMQ yet. > > @Christopher: I'm aware of scheduling feature, but that is more of a > Delayer rather than Throttler. It can delay messages, but cannot > ensure throttled consumption. If I get a burst of messages delay them, > at a later point in time there still will be a burst of messages. > Nothing has improved. > > @Tim: I have described your suggestion with two queues and Camel in > between in as a temporary solution in my recent Camel Design Patterns > book, but that solution is not ideal either. It is still possible to > have messages accumulate in queueu B (for example if the consumers are > stopped temporarily) and then have non throttled consumption breaking > SLAs, etc. > > @Rau I thoughtl PFC should take care of the message backlog and > protect the broker regardless of the reason for message accumulation, > whether that is slow consumers, throttled consumption or consumers > being down for some reason. Having prefetch limit of 1 is good > suggestion though, thanks. > > > > On 30 January 2016 at 21:30, Tim Bain <[email protected]> wrote: > > You could achieve your goal by having two queues, A and B. Have > producers > > publish to A and consumers consume from B, and use an embedded Camel > route > > to move messages from A to B at the desired rate. > > > > But Raul's right: if you're going to do this, make sure there's some kind > > of safety feature to prevent messages from backlogging on the broker. > > Maybe you get it for free based on the characteristics of your clients, > or > > maybe that's some mechanism to discard built-up messages (based on time, > > count, bytes, etc.), or a way to detect a backlog and temporarily speed > up > > the Camel route or to let consumers consume directly from A for a while, > > or... > > > > Tim > > On Jan 30, 2016 12:56 PM, "Raul Kripalani" <[email protected]> wrote: > > > >> Just one remark. This could be dangerous as it could lead to an > >> ever-growing backlog of messages in AMQ. I suppose you have already > >> considered it, and either the producers are slower than the consumers, > or > >> you have multiple competing consumers, or there is a period in the day > >> where the producers stop producing to give the consumers time to consume > >> the existing backlog or something. > >> > >> If you implement this, I would reduce the prefect buffer to 1 if you > have > >> multiple competing consumers to achieve decent distribution and avoid > >> starvation. > >> > >> Raúl. > >> On 30 Jan 2016 18:15, "Bilgin Ibryam" <[email protected]> wrote: > >> > >> > Hi all, > >> > > >> > Apologoes if this is a stupid question. But I wonder is there a way to > >> > do throttling on the consumer side. Let me clarify what I mean. > >> > > >> > PFC can do throttling for the producer, but I need a similar behaviour > >> > on the consumer. For example I want to configure a message consure or > >> > message queue so that messages are consumed/released one per second. > >> > Usually the messages are consumed as fast as the cosnumers can take. > >> > Then I can do some throttling in consumer Camel route, but that > >> > throutling in Camel cannot be clustered accross instances. If the > >> > throttling can be configured ot queue level, then I don't have to do > >> > consumer throttling. > >> > > >> > Thanks, > >> > > >> > -- > >> > Bilgin Ibryam > >> > Camel Committer at ASF & Integration Architect at Red Hat > >> > Blog: http://ofbizian.com | Twitter: @bibryam > >> > > >> > Camel Design Patterns https://leanpub.com/camel-design-patterns > >> > Instant Apache Camel Message Routing > http://www.amazon.com/dp/1783283475 > >> > > >> > > > > -- > Bilgin Ibryam > Camel Committer at ASF & Integration Architect at Red Hat > Blog: http://ofbizian.com | Twitter: @bibryam > > Camel Design Patterns https://leanpub.com/camel-design-patterns > Instant Apache Camel Message Routing http://www.amazon.com/dp/1783283475 >
