I'm sure it's completely solvable, the only question is why you want python in the first place.
Here's a little trick I did that boosted my own performance a lot: my setup is a PHP API server that handles REST requests, and is a thrift client. It passes the parameters to thrift, serializes the response back to JSON, and sends it to the user. The main problem was the python GIL that bounds all threads to a single CPU Core So I did a very simple thing: since my server is mostly stateless, what it does is use the python multiprocessing module to spawn 4-6 thread pool servers on consecutive ports. the php client selects a random port, and I'm utilizing the CPU much better and Increased the RPS by a lot. Another thing with python is, that you can optimize CPU heavy parts of your program and offload them to C code, either by loading a DLL, by building a binary module (like the TBinaryProtocolAccelerated), or by using some C precompiler like Cython. I wonder how difficult it would be to wrap the whole thrift server as a python module and have only the handler written in python. On Mon, Jan 24, 2011 at 5:03 PM, dean forever <[email protected]>wrote: > thanks , Dvir! > finally ,by testing the server, I found the server didnot perform well.Only > 10% CPU usage,comparing to 100% CPU usage of Java Tomcat > I think i need to find a better server program instead of the thrift > default > server. > > > 2011/1/21 Dvir Volk <[email protected]> > > > I don't know about the compact transport, but just a python server tip - > > use TBinaryProtocolAccelerated, handles serialization with C code which > > makes it faster. > > Also, try not to have more jmeter threads than max threads on the server. > > I'm currently testing thrift with jmeter via a PHP proxy rest server, but > > these tips should apply when testing directly with java, I guess. > > > > On Fri, Jan 21, 2011 at 7:23 AM, dean forever < > [email protected] > > >wrote: > > > > > When I use the default example settings for java client and python > > server( > > > like tutorial shows) to do Jmeter Tests,lots of errors such as > > > "TTransportException: Socket already connected" "'Connection reset by > > > peer" > > > "Broken pipe" > > > happened > > > > > > when i try TFramedTransport TCompactProtocol in java, it's all right > > > > > > in python when I use TFramedTransport ,it shows that no listen and > > accept > > > method in it.When I added them by > > > def listen(self): > > > return self.__trans.listen() > > > > > > the python server can run normally > > > > > > > > > but when I start Java Client to request, the python server show lots of > > > NameError in TCompactProtocol.py > > > > > > Anyone has a better solution to this combine???Please help me find a > > best > > > solution for java Client and python Server maintaining > > > high concurrents > > > > > >
