Hi Youcef, WSO2 Message broker has a thread pool with threads working to push messages from DB to subscribers. Thus if we create x number of connections to a broker those x number of connections will be distributed across the threads in the pool. We use LMax Disruptor  to batch incoming messages and write them to DB as batches to gain performance.
There will be a upper bound for number of threads we can register to push messages out. Also, I think one node can handle around 200-500 connections (subscribers). We have not done proper stress testing on that side still. Doing maths and assuming liner scaling, this will need 1M/500 = 2000 nodes in cluster! We cannot maintain such large cluster. Thus suggestion would be to have a small set of clusters pointed to different DBs. My figures might be wrong. Maybe a MB node can handle more connections than that. As a part of evaluation effort this is something needs to be done. If these individual connections are idle most of the time, yes, then can have more connections connected to a single node as threads would be idle and available. Another figure to consider is message distribution algorithm of MB needs to make copies of metadata+content of a single message one per node (if y number of subscribers are there for that topic on that node, that y number of subscribers will share that copy). Thus when number of nodes in a cluster grows, this overhead also grows. This is also something to consider. Cannot say the requirement cannot be achieved, but we need to do a proper evaluation as we have not tested MB on that number of connections. . https://lmax-exchange.github.io/disruptor/ Thanks On Fri, Feb 2, 2018 at 6:41 PM, Youcef HILEM <youcef.hi...@laposte.fr> wrote: > Hi Imesh, > > Thank you very much. > > I must present the solutions to the architectural committee of Tuesday, > 06/02/2018. > It's good to have some information even this weekend on my work email. > > > > Thanks > Youcef HILEM > > > > -- > Sent from: http://wso2-oxygen-tank.10903.n7.nabble.com/WSO2- > Architecture-f62919.html > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- *Hasitha Abeykoon* Associate Technical Lead; WSO2, Inc.; http://wso2.com *cell:* *+94 719363063* *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com>
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture