i haven't seen this before but what i can suggest you is these 2 steps and you will probably get more insights - 1) # bin/hbase shell # list
see if the list commands returns back with your list of tables. 2) take a look at the latest hbase logs and see if there are any FATAL, ERROR or WARN logs lately. thanks On Fri, Oct 7, 2011 at 1:31 AM, Joel Halbert <[email protected]> wrote: > Hi, > > Intermittently I am seeing my client threads stuck waiting on a call to > HBaseClient.call(HBaseClient.java:722) > > This is a stand-along instance of HBase (no Hadoop). The client > processes that use it are multi-threaded, and I am using a HTablePool to > get HTable instances. > > Has anyone seen this before? Any ideas what could be the cause? > > A thread dump of a client process shows: > > e.g. > UPDATE_CLIENT_SEEDS" prio=10 tid=0x09fbcc00 nid=0x34e0 in Object.wait() > [0x0ae5a000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:485) > at > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:722) > - locked <0x30bdd308> (a org.apache.hadoop.hbase.ipc.HBaseClient > $Call) > at org.apache.hadoop.hbase.ipc.HBaseRPC > $Invoker.invoke(HBaseRPC.java:333) > at $Proxy0.get(Unknown Source) > at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:450) > at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:448) > at org.apache.hadoop.hbase.client.HConnectionManager > $TableServers.getRegionServerWithRetries(HConnectionManager.java:1040) > at org.apache.hadoop.hbase.client.HTable.get(HTable.java:447) > > > "R-PPQ-6" daemon prio=10 tid=0x0738e000 nid=0x3510 in Object.wait() > [0x04ac4000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:485) > at > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:722) > - locked <0x155328e8> (a org.apache.hadoop.hbase.ipc.HBaseClient > $Call) > at org.apache.hadoop.hbase.ipc.HBaseRPC > $Invoker.invoke(HBaseRPC.java:333) > at $Proxy0.get(Unknown Source) > at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:450) > at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:448) > at org.apache.hadoop.hbase.client.HConnectionManager > $TableServers.getRegionServerWithRetries(HConnectionManager.java:1040) > at org.apache.hadoop.hbase.client.HTable.get(HTable.java:447) > at > com.su3analytics.sitedelta.pages.PageTable.get(PageTable.java:111) > at > com.su3analytics.sitedelta.pages.PageTable.get(PageTable.java:256) > > > "R-PPQ-9" daemon prio=10 tid=0x09ded400 nid=0x354a in Object.wait() > [0x019fe000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:485) > at > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:722) > - locked <0x1576d880> (a org.apache.hadoop.hbase.ipc.HBaseClient > $Call) > at org.apache.hadoop.hbase.ipc.HBaseRPC > $Invoker.invoke(HBaseRPC.java:333) > at $Proxy0.get(Unknown Source) > at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:450) > at org.apache.hadoop.hbase.client.HTable$4.call(HTable.java:448) > at org.apache.hadoop.hbase.client.HConnectionManager > $TableServers.getRegionServerWithRetries(HConnectionManager.java:1040) > at org.apache.hadoop.hbase.client.HTable.get(HTable.java:447) > > > > >
