> Not to mention the fact that inevitably, you probably are going to want some > security on those connections, which means TLS, which means individual crypto > connections.
If there is a need for encryption, then yes, .. but it's not always needed > > I believe the best model for this kind of high-volume reliable-multicast-over- > unicast is a spanning tree, like what IRC uses for server-to-server Ok. Makes sense. I see .. could have frontend nodes, which act as aggregators condensing the messages on many incoming connections into 1 uplink TCP connection to a backend node .. or a hierarchy. This is quite interesting .. for I am now beginning to see why Google is pushing the discussion on WebSocket multiplexing extension on Hybi;) Until now, I thought that would be primarily because Chrome has a process per tab design .. but that's probably only one reaon. The cloud side being the other one .. > communication. If you have 100,000 concurrent, active connections, you're > already probably beyond the CPU constraints of a single machine. So have your Not really. I've been testing up to 180k connections on a 2 core virtual machine and when doing PubSub over those (dispatching 1 msg to all 180k) .. CPU load isn't maxed out .. but I'm hitting a limit of around 20k/s dispatched msgs. My suspicion is it's probably more interrupt/syscall overhead bound .. > clients connect (via round-robin DNS, or whatever other mechanism makes > sense) to a variety of different servers, then have the servers connected to > each > other in a pattern such that one server tells ten friends, and they each tell > ten > friends, and so on, until each server is only responsible for spewing out > data to > a reasonable number of connections (5000 maybe?). 5k is much too low .. would require too many servers. My target is 200k per node and have as high as possible msgs/s dispatch rate. Probably a design with frontend nodes aggregating TCP connections is inevitable .. Thanks! _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
