What we have been seeing with our testing is a linear decrease in delivery rate per consumer as we increase the number of consumers
e.g. 1,600 m/m with 1 consumer, 800 m/m with 2, 400 m/m with 4, 320 m/m with 5, etc.. Note that these messages are being delivered over the internet, and we are using Pub/Sub with Topics, The producer machine is a pretty nice/quick machine - and we don't see any bottlenecks there, the bottleneck seems to be in our T1 connection. but to make this scale to 200-250 consumers, and maintain as least 200 m/m per consumer - it seems we would need A LOT of bandwidth ! hence the reason for looking at multicast / network of brokers / JXTA / etc.. I thought _maybe_ JXTA and its PIPEs could be configured in such a way as to give a psuedo/virtual multicast using a tree of nodes. or do the same thing with ActiveMQ brokers -- but again -- we don't want to have to setup every node by hand. any other ideas ? James.Strachan wrote: > > Am not aware of anyone who uses multicast over the internet - even > JXTA is mostly socket based AFAIK, it just deals with NAT issues I > thought but using regular sockets? > > So I'd just use a couple of brokers. I guess you're using topics > right? A single broker should be able to handle 1000s of clients with > hundreds of messages a second per consumer on a fast network & decent > PC with a reasonable disk. > > If you've only 1 producer, you might wanna use async dispatch to get > the best throughput across hundreds of clients for a single producer. > > What kind of throughput do you get in your environment when you try > hundreds of clients? > > > On 11/24/06, terrytriple <[EMAIL PROTECTED]> wrote: >> >> Hi There, >> >> Currently we are using JBossMQ to send data in a pub/sub model to a few >> consumers, >> over the internet - our testing has shown ActiveMQ to be significantly >> faster than JBossMQ : >> >> 600-700 2k messages / minute with JBossMQ and 1,600 2k messages / minute >> with ActiveMQ. >> - 1 Publisher (pretty good / fast machine - 2 x 3GHz, 2GB Ram, RAID 10, >> etc) >> - Publisher in on a T1 Line >> - Consumers are on DSL >> - Sending messages over the internet >> - Messages are Persistant, and Transactional >> >> we need to sustain at least 200 messages / minute per client, the above >> figures are an aggregate figure, >> e.g. 600-700 m/m with JBossMQ equates to 200-240 m/m for 3 consumers, >> >> However - we need this to upscale this solution from 4-5 consumers to 200 >> consumers, >> very soon -- and based on the tests we have done - we would need to >> invest >> in lots of >> bandwith and several servers to get this to work. >> >> e.g. 200 consumers x 200 m/m = 40,000 m/m total >> >> so -- this being the case - we are looking for alternative solutions, >> >> we have been looking at using multicast over the internet, >> but information seems sketchy, >> and although I have tried speaking to many ISPs about it >> - It's difficult to get anyone who knows what I'm talking about, >> and some simply say no. >> >> besides the fact it seems AMQ 4 does not seem to support multicast >> delivery >> ? (only discovery). >> >> would multicast via JGroups or JXTA work ? do these work ? >> >> I believe JXTA would/could do some sort of Virtual / Application Level >> Multicast and perhaps organise the nodes into a tree and delivery >> messages >> that way ? >> >> we like the idea of using a "Network of Brokers", or setting up some kind >> of >> tree, >> where our central server sends the messages to a few other brokers, and >> then >> more brokers collect from those. >> but we don't really want to have to go round each server and manually >> configure the connections. >> >> could the configuration be made more dynamic ? would it be possible to >> send >> configuration changes via JMS ? >> >> Anyone got any advice / thoughts ? >> >> >> Many Thanks >> >> -- >> Terry Paterson >> Actual Systems UK Ltd >> >> -- >> View this message in context: >> http://www.nabble.com/Multicast---Application-Level-Multicast---JXTA---tf2699960.html#a7528866 >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> >> > > > -- > > James > ------- > http://radio.weblogs.com/0112098/ > > -- View this message in context: http://www.nabble.com/Multicast---Application-Level-Multicast---JXTA---tf2699960.html#a7540845 Sent from the ActiveMQ - User mailing list archive at Nabble.com.
