The TThreadPoolServer components are thread-safe. However, any given
client for a service is NOT thread-safe. This means that if you are
using multiple threads on the client side, you either need multiple
connections or application-level thread safety.
-Bryan
On Nov 22, 2009, at 11:09 AM, kpowerinfinity wrote:
Hello,
We've been using Thrift for over a year now and have been very happy
with how it makes it easy to implement RPC across languages. However,
we were facing a perf bottleneck since each call was taking about
50-75ms which means we can't make more than 50k calls in an hour. We
are using this for a message broker and want to be able to send out
millions of messages per hour. This was on a single thread. On trying
out the most obvious multi-threaded implementation where we pump out
the calls on multiple-threads, I get the exceptions below.
What I am not sure about is why thrift has an out of memory problem on
the server (thrift-handler) end when I am pumping in messages on
multiple-threads from the client. I have a TThreadPoolServer on the
server end.
Is there something I am missing?
Regards
Krishna.
Exception in thread "Thread-2" java.lang.OutOfMemoryError: Java
heap space
at com.facebook.thrift.protocol.TBinaryProtocol.readStringBody
(Unknown Source)
at com.facebook.thrift.protocol.TBinaryProtocol.readString(Unknown
Source)
at com.facebook.thrift.protocol.TBinaryProtocol.readMessageBegin
(Unknown
Source)
at in.capillary.ifaces.nsadmin.NSAdminService
$Client.recv_processMessageFromClient(NSAdminService.java:154)
at in.capillary.ifaces.nsadmin.NSAdminService
$Client.processMessageFromClient(NSAdminService.java:139)
at in.capillary.msging.system.NSAdminClientAdapter.send
(NSAdminClientAdapter.java:108)
at in.capillary.msging.system.NotificationDeliveryClient.send
(NotificationDeliveryClient.java:42)
at
in.capillary.msging.system.ThreadWorker.retrieveAndSendSingleMessage
(ThreadWorker.java:64)
at in.capillary.msging.system.ThreadWorker.makeDeliveriesTillStop
(ThreadWorker.java:82)
at in.capillary.msging.test.TestShopbookPerf$3.run
(TestShopbookPerf.java:108)
at java.lang.Thread.run(Thread.java:619)
00:24:21,092 INFO [Thread-0] NSAdminClientAdapter:80 - Registering
with NSAdmin
00:24:21,094 ERROR [Thread-0] NSAdminClientAdapter:24 - Exception
thrown: com.facebook.thrift.TApplicationException:
processMessageFromClient failed: unknown result
At:
NSAdminService.java:in.capillary.ifaces.nsadmin.NSAdminService
$Client:recv_processMessageFromClient:169
NSAdminService.java:in.capillary.ifaces.nsadmin.NSAdminService
$Client:processMessageFromClient:139
NSAdminClientAdapter.java:in.capillary.msging.system.NSAdminClientAdap
ter:send:108
NotificationDeliveryClient.java:in.capillary.msging.system.Notificatio
nDeliveryClient:send:42
ThreadWorker.java:in.capillary.msging.system.ThreadWorker:retrieveAndS
endSingleMessage:64
ThreadWorker.java:in.capillary.msging.system.ThreadWorker:makeDeliveri
esTillStop:82
TestShopbookPerf.java:in.capillary.msging.test.TestShopbookPerf
$1:run:86
Thread.java:java.lang.Thread:run:619
com.facebook.thrift.TApplicationException: processMessageFromClient
failed: unknown result
at in.capillary.msging.system.NSAdminClientAdapter.send
(NSAdminClientAdapter.java:111)
at in.capillary.msging.system.NotificationDeliveryClient.send
(NotificationDeliveryClient.java:42)
at
in.capillary.msging.system.ThreadWorker.retrieveAndSendSingleMessage
(ThreadWorker.java:64)
at in.capillary.msging.system.ThreadWorker.makeDeliveriesTillStop
(ThreadWorker.java:82)
at in.capillary.msging.test.TestShopbookPerf$1.run
(TestShopbookPerf.java:86)
at java.lang.Thread.run(Thread.java:619)
--
http://www.linkedin.com/in/kpowerinfinity
Note: this is the original facebook implementation of thrift.
