Which protocol do you use ? During the observed performance hit, memory is sufficient and CPU is fully utilized ?
If you're using binary protocol, TBinaryProtocolAccelerated class is an option. It uses C extension and in my past local test, was a lot faster than pure python counter part. Unfortunately, there's a known problem with large collections in latest 0.9.3 so you may need to downgrade to 0.9.2 with this one. There are also a few generator switches to change the run-time behavior like -gen py:dynamic etc. Some of them claim that they dramatically reduce memory consumption. On Sun, Dec 20, 2015 at 5:19 PM 刘畅 <[email protected]> wrote: > The background is that, we used to use Pyro4 to do the magic of RPC > between our two Python processes. > > The serializer is cPickle in Pyro4. However, in our new approach, we > change one of the process into C++ using Thrift instead. > > The performance when sending small request/response, Thrift is doing fine. > > But in one of the cases, the size of the response is over 200 KB, and the > performance is dropping rapidly. > > Our implementation detail is listed below: > > Server is using C++, Client is using Python, using TFramedTransport as the > transportation and TThreadedServer as server. > > Unfortunately, our project cannot stand this performance drop between two > version. And I’m here for help. > > I’d like to know whether there is any performance improvement solution in > serialization?
