Hi Ted, The region servers are not loaded. It is showing 5% CPU usage. The datanode is showing around 50% CPU utilization. disk IO is aroung 7Mbps.
There is nothing noticeable in GC log. Thanks, Anand On 12 April 2013 02:56, Ted Yu <[email protected]> wrote: > How loaded were the region servers when the query was running ? > > Did you check GC log ? > > Thanks > > On Thu, Apr 11, 2013 at 8:23 AM, anand nalya <[email protected]> > wrote: > > > Hi, > > > > I'm using HBase 0.94.5 with thrift server. I'm trying to get the rows > from > > HBase using > > org.apache.hadoop.hbase.thrift.generated.Hbase.Client.getRows(ByteBuffer, > > List<ByteBuffer>, Map<ByteBuffer, ByteBuffer>) but it is giving results > > very slowly (around 2 mins for 100 rows). For larger number of records, > > there is no response. > > > > I've two region server and a total of 128 regions. Total data size is > > around 250GB (250 million records) uniformly distributed across regions. > > > > Regionserver only show the following in its log: > > > > 2013-04-11 19:53:44,535 ERROR > > org.apache.hadoop.hbase.regionserver.HRegionServer: > > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call > > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc > version=1, > > client version=29, methodsFingerPrint=-1368823753 from > > 192.168.145.195:52277after 74994 ms, since caller disconnected > > at > > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664) > > at > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576) > > at > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > > at > > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > > 2013-04-11 19:53:46,121 ERROR > > org.apache.hadoop.hbase.regionserver.HRegionServer: > > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call > > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc > version=1, > > client version=29, methodsFingerPrint=-1368823753 from > > 192.168.145.195:52277after 76580 ms, since caller disconnected > > at > > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664) > > at > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576) > > at > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > > at > > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > > > > > > Any idea what might be wrong here? > > > > Thanks, > > Anand > > >
