Hi Youcef, WSO2 Message broker has a thread pool with threads working to push messages from DB to subscribers. Thus if we create x number of connections to a broker those x number of connections will be distributed across the threads in the pool. We use LMax Disruptor [1] to batch incoming messages and write them to DB as batches to gain performance.
There will be a upper bound for number of threads we can register to push messages out. Also, I think one node can handle around 200-500 connections (subscribers). We have not done proper stress testing on that side still. Doing maths and assuming liner scaling, this will need 1M/500 = 2000 nodes in cluster! We cannot maintain such large cluster. Thus suggestion would be to have a small set of clusters pointed to different DBs. My figures might be wrong. Maybe a MB node can handle more connections than that. As a part of evaluation effort this is something needs to be done. If these individual connections are idle most of the time, yes, then can have more connections connected to a single node as threads would be idle and available. Another figure to consider is message distribution algorithm of MB needs to make copies of metadata+content of a single message one per node (if y number of subscribers are there for that topic on that node, that y number of subscribers will share that copy). Thus when number of nodes in a cluster grows, this overhead also grows. This is also something to consider. Cannot say the requirement cannot be achieved, but we need to do a proper evaluation as we have not tested MB on that number of connections. [1]. https://lmax-exchange.github.io/disruptor/ Thanks On Fri, Feb 2, 2018 at 6:41 PM, Youcef HILEM <[email protected]> wrote: > Hi Imesh, > > Thank you very much. > > I must present the solutions to the architectural committee of Tuesday, > 06/02/2018. > It's good to have some information even this weekend on my work email. > > > > Thanks > Youcef HILEM > > > > -- > Sent from: http://wso2-oxygen-tank.10903.n7.nabble.com/WSO2- > Architecture-f62919.html > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- *Hasitha Abeykoon* Associate Technical Lead; WSO2, Inc.; http://wso2.com *cell:* *+94 719363063* *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
