Hi Azuryy, I'm using the default cache size of 100 for scanner. For mutigets, I've tried with 1 (13ms), 10(356ms), 100(1135ms), 1000(4330ms), and 10000(17744ms) keys. Normal workload will be around 10000 keys at a time.
Are there any optimization that can be done for multigets. Is HBase a good candidate for usecase? Thanks, Anand On 12 April 2013 19:17, anand nalya <[email protected]> wrote: > Hi Ted, > > The region servers are not loaded. It is showing 5% CPU usage. The > datanode is showing around 50% CPU utilization. disk IO is aroung 7Mbps. > > There is nothing noticeable in GC log. > > Thanks, > Anand > > > On 12 April 2013 02:56, Ted Yu <[email protected]> wrote: > >> How loaded were the region servers when the query was running ? >> >> Did you check GC log ? >> >> Thanks >> >> On Thu, Apr 11, 2013 at 8:23 AM, anand nalya <[email protected]> >> wrote: >> >> > Hi, >> > >> > I'm using HBase 0.94.5 with thrift server. I'm trying to get the rows >> from >> > HBase using >> > >> org.apache.hadoop.hbase.thrift.generated.Hbase.Client.getRows(ByteBuffer, >> > List<ByteBuffer>, Map<ByteBuffer, ByteBuffer>) but it is giving results >> > very slowly (around 2 mins for 100 rows). For larger number of records, >> > there is no response. >> > >> > I've two region server and a total of 128 regions. Total data size is >> > around 250GB (250 million records) uniformly distributed across regions. >> > >> > Regionserver only show the following in its log: >> > >> > 2013-04-11 19:53:44,535 ERROR >> > org.apache.hadoop.hbase.regionserver.HRegionServer: >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call >> > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc >> version=1, >> > client version=29, methodsFingerPrint=-1368823753 from >> > 192.168.145.195:52277after 74994 ms, since caller disconnected >> > at >> > >> > >> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664) >> > at >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576) >> > at >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> > at >> > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> > at java.lang.reflect.Method.invoke(Method.java:597) >> > at >> > >> > >> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) >> > at >> > >> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) >> > 2013-04-11 19:53:46,121 ERROR >> > org.apache.hadoop.hbase.regionserver.HRegionServer: >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call >> > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc >> version=1, >> > client version=29, methodsFingerPrint=-1368823753 from >> > 192.168.145.195:52277after 76580 ms, since caller disconnected >> > at >> > >> > >> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664) >> > at >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576) >> > at >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> > at >> > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> > at java.lang.reflect.Method.invoke(Method.java:597) >> > at >> > >> > >> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) >> > at >> > >> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) >> > >> > >> > Any idea what might be wrong here? >> > >> > Thanks, >> > Anand >> > >> > >
