I think perhaps you misunderstood what I was recommending. You shouldn't need to adjust any client code. You certainly don't *need* to create a consumer on every node of the cluster as you imply. Using ON_DEMAND with a redistribution-delay > 0 will allow clients to connect to any node of the cluster and consume messages sent to any other node in the cluster - at any time.
My point was that in order to optimize performance (i.e. the whole point of clustering in the first place) then you should to size your cluster based on the actual client load. To reiterate, if you don't have enough clients to avoid moving messages between cluster nodes then your cluster is likely too large. Another way to think about it is that if consumers are starving then your cluster is likely too large. Again, this goes back to using your resources effectively and efficiently. This is especially important in cloud use-cases where you may be paying by the hour to use a machine that isn't really necessary, but it's also important for bare-metal uses-cases as well to avoid expenditure to acquire the nodes in the first place. The simplest architecture possible is always preferable as it reduces costs for development, deployment, and maintenance. If you're using a cluster of 3 pairs simply to establish a quorum to avoid split-brain with replication then switch to the pluggable quorum voting and use ZooKeeper instead. Justin On Mon, Apr 10, 2023 at 11:57 AM prateekjai...@gmail.com < prateekjai...@gmail.com> wrote: > Hi Justin, > > Thanks for replying. There is a reason why I don't want to create a > consumer per broker/instance of artemis. I am trying to come up with an > architecture for a product where an artemis cluster can expand or shrink > but it shouldn't have any impact on client code. > > Considering the suggested approach, client code has to be updated according > to the size of the cluster. So, I was thinking, could this be possible that > client can connect to any of the broker and then messages could be routed > because consumers might not always be online. Consumers can connect once in > a while. This case becomes especially important, while upgrading clusers. > > Regards, > Prateek Jain > > -------------------------------------------------------------- > EXPECTATION : Causes all troubles...... > -------------------------------------------------------------- > > > On Mon, Apr 10, 2023 at 4:39 PM Justin Bertram <jbert...@apache.org> > wrote: > > > While it's true that with ON_DEMAND you don't get the same behavior as > > STRICT (as one would expect), you still get the benefit of load-balancing > > because messages will be initially distributed to nodes that have > > consumers. This is why it's called "on demand" - messages are distributed > > to where the consumers are rather than strictly round-robined across all > > cluster nodes. > > > > You can achieve 1, 2, & 3 with ON_DEMAND and a redistribution-delay > 0 > > [1]. This is the most common configuration. > > > > That said, clustering is all about increasing overall message throughput > > via horizontal scaling. In order to optimize performance you really don't > > ever want to move messages between nodes as that adds latency. You want > > every node in the cluster to have enough consumers to process all the > > messages sent to that node. If that's not the case that's an indication > > that the cluster is, in fact, too large and you're wasting resources. I > > recently added a new section to the cluster documentation [2] discussing > > this very thing. > > > > > > Justin > > > > [1] > > > > > https://activemq.apache.org/components/artemis/documentation/latest/clusters.html#message-redistribution > > [2] > > > > > https://github.com/apache/activemq-artemis/blob/main/docs/user-manual/en/clusters.md#performance-considerations > > > > On Fri, Apr 7, 2023 at 7:51 AM prateekjai...@gmail.com < > > prateekjai...@gmail.com> wrote: > > > > > Hi Roskvist, > > > > > > I tried the ON_DEMAND value but still it doesnt work. Infact, with > > > on_demand value the loadbalancing stops and the whole scalability > feature > > > in the cluster becomes irrelevant. > > > > > > IMO, if the client for a queue/topic is connected to any of the broker > > > instances then; messages should get routed to it. So, in nutshell; > what I > > > am trying to achieve here is - > > > > > > 1. Deploy cluster in such a way that it should be easy to scale. > > > 2. Scalability should be transparent to the client code. Clients should > > > only know about broker IPs and Ports. > > > 3. A Client should be able to send and receive messages w/o taking into > > > consideration; to which broker they are connected to. > > > > > > I am able to achieve most of them. It is only the receiving part which > is > > > not working as desired. In the clustered queue example, something very > > > similar is achieved but it requires, client to be connected to both > > > instances. But in my case, I am trying to achieve it by connecting to > any > > > instance (live) in the cluster. > > > > > > Regards, > > > Prateek > > > > > > > > > On Fri 7 Apr 2023, 12:56 Roskvist Anton, <anton.roskv...@volvo.com> > > wrote: > > > > > > > Right, so as far as I am aware the STRICT load balancing policy does > > not > > > > allow for message redistribution, it's purpose is to divide incoming > > > > messages evenly across the cluster regardless of client/consumer > state. > > > > Perhaps ON_DEMAND might be better suited for your needs, or possibly > > > > OFF_WITH_REDISTRIBUTION and handling initial distribution of messages > > via > > > > client side load balancing? > > > > > > > > ________________________________ > > > > > > > > This email message (including its attachments) is confidential and > may > > > > contain privileged information and is intended solely for the use of > > the > > > > individual and/or entity to whom it is addressed. If you are not the > > > > intended recipient of this e-mail you may not disseminate, distribute > > or > > > > copy this e-mail (including its attachments), or any part thereof. If > > > this > > > > e-mail is received in error, please notify the sender immediately by > > > return > > > > e-mail and make sure that this e-mail (including its attachments), > and > > > all > > > > copies thereof, are immediately deleted from your system. Please > > further > > > > note that when you communicate with us via email or visit our website > > we > > > > process your personal data. See our privacy policy for more > information > > > > about how we process it: > https://www.volvogroup.com/en-en/privacy.html > > > > > > > > > >
