On 09/08/12 09:18, Akshat Aranya wrote:
On Wed, Aug 8, 2012 at 5:10 PM, Matthew Chambers<[email protected]> wrote:
Thanks for the information, Mark. This is not a criticism of Thrift,
but sometimes having more threads than the number of cores is
desirable when you have a thread pool with worker threads doing
network/disk I/O. I have programmed with both a message passing model
and a worker thread model, and in my opinion, the latter results in
more readable code and is easier to program with, especially with
languages such as C++ that require explicit memory management. I
suppose it is possible to keep most of code in a worker thread model
and just use a queue of requests at the lowest level where I need to
make Thrift RPC calls. I was hoping that Thrift would do this for me
out of the box. :-D
Cheers,
Akshat
If that is what your doing, then you might want to create a pool of thrift
connections your threads can check out, work with, and then check back in.
In this case, its often helpful to have a super lightweight service method
that can be used to test the connection before giving it to the thread, so
you can reconnect if necessary.
-Matt
That still doesn't allow me to make a synchronous remote call from my
application logic thread, which is much easier to program than asking
another thread to make the remote call. For now, I've implemented
ThreadLocal Clients for my application logic threads, each with their
own Transports, which means each thread has its own connection. I
don't know yet if that'll be a scalability issue, especially on the
server where multiple multi-threaded clients will connect to.
Cheers,
Akshat
I don't see why not, your application thread is just another thread that
can do something like:
service = thiftPool.checkOutService()
try:
do stuff
finally:
thriftPool.checkIn(service)
Or, give your application thread its own instance of the service at
start up.
Sockets are not thread safe. That is how its always been. Sharing a
resource like a socket between threads slows it down, it doesn't speed
it up. I doubt you'll see scalability issues. I don't know how many
clients you plan to have, but I work with in the 10-30k range
(admittedly, not that many) You might have to increase the default
number of allowed open file handles on your server depending on how many
nodes you have, but with thrift is far easier to saturate your network
connection long before you saturate the CPUs.
-Matt
