Hi Shammi, On Fri, Sep 29, 2017 at 4:18 AM, Shammi Jayasinghe <[email protected]> wrote:
> > > On Thu, Aug 17, 2017 at 4:15 AM, Asanka Abeyweera <[email protected]> > wrote: > >> Hi all, >> >> I am working on implementing active-passive clustering on top of the new >> MB4 architecture. Current message delivery implementation in MB4 is not >> capable to work in a multi-node environment. If we run more than one node >> concurrently, messages can be duplicated since all nodes start delivering >> messages available for a queue in the message store. We have to make the >> MB4 nodes cluster aware to rectify this behavior. >> High-level design >> >> We are planning to use the available RDBMS leader election mechanism to >> decide on the active node. All the non-leader nodes will be passive nodes >> and will reject all incoming requests. The clients should be configured >> with failover URLs so that they can failover to the active node from >> passive nodes. >> > > Hi Asanka, > > Sorry for the late reply. AFAIU, with this new design, although there are > multiple message broker nodes in the deployment, Only one node will be > working as the active node and others will be idle until they get selected > as the ACTIVE node. > > In that kind of behavior, Are we deviating from the concept of distrubited > message brokering? > > When there is a high load (Thousands of subscriptions or consumers/ > Thousands of publishers or producers), All this node is going to be > applying on one node which is going to use high CPU power and memory. In > that case, we have to define the numbers which can be handled with specific > memory and CPU power of a box. If there is a higher load, it will increase > the delay of the message delivery also. How are we going to address these > in this new design? > We are bit deviating from the distributed queueing concept. The plan is to bind a single queue to a single node and make all subscribers and publishers to a queue connect to one broker. The reason for the decision is, in practice a given queue tends to have only a small number of subscribers (~10). Therefore we can avoid doing distributed coordination per message which ultimately result in more simplified and performant architecture. As the first iteration we will be focusing on releasing an active-passive clustering implementation. But the long term plan will be the one I described previously. > Thanks > shammi > >> >> Tasks to do when the active node becomes passive >> >> 1. >> >> Enable rejecting all requests to broker node except the CLOSE-OK >> 2. >> >> Close existing AMQ connections (both publishing and consuming) >> >> >> Tasks to do when the passive node becomes active >> >> 1. >> >> Start accepting all requests to broker node >> >> >> The assumption here is the active components like the Inbound disruptor, >> outbound disruptor, delivery task managers will stop working (consuming >> resources) when the incoming requests are stopped and subscriptions are >> removed in passive nodes. >> >> >> >> - >> >> Filtering message events will be handled by “InboundEventGatekeeper”. >> - >> >> DeliveryHandler will also be modified to stop delivering messages >> when a node becomes passive. >> - >> >> All the connections will be closed using the server connection >> registry >> >> >> Suggestions and feedback are appreciated. >> >> -- >> Asanka Abeyweera >> Senior Software Engineer >> WSO2 Inc. >> >> Phone: +94 712228648 <+94%2071%20222%208648> >> Blog: a5anka.github.io >> >> <https://wso2.com/signature> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Best Regards, > > * Shammi Jayasinghe* > > > *Senior Technical Lead* > *WSO2, Inc.* > *+1-812-391-7730 <(812)%20391-7730>* > *+1-812-327-3505 <(812)%20327-3505>* > > *http://shammijayasinghe.blogspot.com > <http://shammijayasinghe.blogspot.com>* > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Asanka Abeyweera Associate Technical Lead WSO2 Inc. Phone: +94 712228648 Blog: a5anka.github.io <https://wso2.com/signature>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
