increasing threads would increase performance - but of course it's subject to limitations of what hbase end can do.
sometime recently, we had run a similar benchmark (4 RS, 16 disks) and random iops over a very large dataset scaled from 85 op/s...@~10ms to 500op/s...@50ms peak. I don't remember the number of threads at the peak (it was about 30-50 i believe). for these sort of benchmarks - we have tried many-thre...@single-machine to few-thre...@multiple-machines - and there's been some difference (in favor of the latter) - but not significant (particularly for random reads). (so client side multithreading issues are not significant for this sort of test). in our case - the data set was large enough that we were bound by the disk reads (by DFS) (iostat showed heavy disk traffic). Depending on ur data size and key locality - the speedup may be more. we weren't using lzo compression (that would have produced better numbers most likely). On Thu, Jan 14, 2010 at 1:39 PM, Boris Aleksandrovsky <balek...@gmail.com> wrote: > I have a moderately large HBase table of about 1M rows distributed across 4 > region servers. I also have a requirement to retrieve a 1000 rows from that > table (one can assume keys are randomly distributed) at the same time. I > would ideally like to have a facility to batch read of all 1000 rows at the > same time, but I do not think HBase has such a facility. The only way I was > able to find is to sequentially retrieve one row at a time using Get row > API. Is there a way to improve on it? > > One way I can think of is to create an HTable object pool and issue > concurrent requests to HBase. This would probably help, but I do not expect > for performance to increase X time, where X is the number of threads in my > application. > > Is there a clever way to batch requests, so at least you can issue multiple > row reads to the same region server at the same time? Any other tricks or > suggestions will be appreciated. > > -- > Thanks, > > Boris > http://twitter.com/baleksan > http://www.linkedin.com/in/baleksan >
