[Twisted-Python] scaling with lots of clients

2011-08-03 Thread Grant Mckenzie
Hi, I have a twisted Perspective Broker based server. Recently with an increased number of clients we are seeing a decrease in throughput. Profiling the server it seems that it is spending the majority of time sending messages to the clients (unsuprisingly). Messages are sent to the client

Re: [Twisted-Python] scaling with lots of clients

2011-08-03 Thread Itamar Turner-Trauring
I have a twisted Perspective Broker based server. Recently with an increased number of clients we are seeing a decrease in throughput. Profiling the server it seems that it is spending the majority of time sending messages to the clients (unsuprisingly). PyPy might speed this up quite a bit;

Re: [Twisted-Python] scaling with lots of clients

2011-08-03 Thread Grant Mckenzie
Threading won't help much if you're CPU bound, since Python can only run one Python thread at once. Process pooling probably would help, though will involve a more complex infrastructure. Which part of the docs made you think that process pooling is a bad idea? Hi Itamar, on re-reading I see