Hbase does do a lot of small RPC and a very preliminary analysis has indicated that is an area with some promise in terms of optimization. I don't expect Thrift to be a magic bullet by any means but it is one of the things we are looking at. When we have some real numbers to share, we'll send pass them along.
Chad On 12/4/07 9:05 AM, "Doug Cutting" <[EMAIL PROTECTED]> wrote: Chad Walters wrote: > 3. More performance work: Michael did some performance measurements a while > back that seemed to indicate a lot of time spent back-and-forth in RPC. We're > exploring Thrift as a lighter-weight RPC mechanism, but there are probably > other things to be done to reduce this cost. More analysis and measurement > would be helpful. Hmmm. Hadoop's RPC has its shortcomings, but I wouldn't call it heavy-weight or low-performance. And Thrift has advantages over Hadoop RPC, but I have not heard that performance is a primary one. So I would not assume that replacing Hadoop RPC with Thrift would improve performance.
