[
https://issues.apache.org/jira/browse/THRIFT-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dvir Volk updated THRIFT-1083:
------------------------------
Attachment: TProcessPoolServer.py
This is a newer version of the server, that adds an optional "post fork
callback" that allows you to initialize state in individual handler processes,
after the handlers have been forked. I used it for example to init redis
connections.
> Preforking python process pool server
> -------------------------------------
>
> Key: THRIFT-1083
> URL: https://issues.apache.org/jira/browse/THRIFT-1083
> Project: Thrift
> Issue Type: New Feature
> Components: Python - Library
> Affects Versions: 0.6
> Environment: linux. haven't tested on windows
> Reporter: Dvir Volk
> Priority: Minor
> Attachments: TProcessPoolServer.patch, TProcessPoolServer.py,
> TProcessPoolServer.py
>
>
> This patch adds a new server to the python library: TProcessPoolServer, which
> is a preforking python server.
> this server is sort of a mix between TForkingServer and TThreadPoolServer: It
> has a pool of workers that are preforked subprocesses.
> This approach allows the user to avoid the python GIL single processor limit
> in threading applications, and use a high performance server, and at the same
> time it avoids the need to fork a child process for each connection, as is
> the case with the forking server.
> I've benchmarked it to be about 5-6 times faster than TThreadPoolServer on a
> quad Corei7 CPU, and about the same amount faster than TForkingServer if you
> are not using persistent connections and forking a child for each request.
> The patch also updates the python unit tests to include tests for this server.
> Notes:
> 1. Of course this server has the limitations of forking regarding shared
> state and memory copying.
> 2. You should NOT kill the server with kill -9 - as this will not allow the
> parent process to terminate its children, resulting in orphaned processes
> that keep your socket open. either run stop() in your app or kill the process
> with SIGINT. It will respond to ctrl+C however
> 3. use setNumWorkers(n) before starting the server to determine how many
> processes you want to spawn. the default is 10.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
