Finally, I found that this is the same issue as https://issues.apache.org/jira/browse/HBASE-7685. I don't know why RpcClient.java still use the code which mentioned in 7685 for connection close method. Is there anyone can tell me the reason?
在 2013-6-24，下午7:06， Xiong LIU 写道： > Hi, all, > when I configure ipc pool in hbase-site.xml as follows: > > <property> > <name>hbase.client.ipc.pool.type</name> > <value>RoundRobinPool</value> > </property> > <property> > <name>hbase.client.ipc.pool.size</name> > <value>1</value> > </property> > > the start and stop processes of hbase run without any problem. > > But if I set hbase.client.ipc.pool.size to 10 (20, 30, or possibly some > other values), some problems occur. > > In the regionserver's log, I found that the regionserver continuously > attempted connect to master server: > > 2013-06-24 16:59:11,734 INFO [regionserver60020] > regionserver.HRegionServer: Attempting connect to Master server > at HOSTNAME-TEST,60000,1372064256847 > 2013-06-24 16:59:14,738 INFO [regionserver60020] > regionserver.HRegionServer: Attempting connect to Master server > at HOSTNAME-TEST,60000,1372064256847 > 2013-06-24 16:59:17,741 INFO [regionserver60020] > regionserver.HRegionServer: Attempting connect to Master server > at HOSTNAME-TEST,60000,1372064256847 > 2013-06-24 16:59:20,744 INFO [regionserver60020] > regionserver.HRegionServer: Attempting connect to Master server > at HOSTNAME-TEST,60000,1372064256847 > ......many more > > About 30 minutes after the master started, in the master's log, > client.ServerCallable began to throw the following exception repeatedly: > > 2013-06-24 17:32:47,233 WARN > [HOSTNAME-TEST,60000,1372064256847-CatalogJanitor] client.ServerCallable: > Call exception, tries=0, numRetries=200 > java.io.IOException: Call to HOSTNAME-TEST/192.168.1.35:60020 failed on > local exception: java.io.IOException: Call 31 not added as the > connection HOSTNAME-TEST/ > 192.168.1.35:60020/ClientService/hadoop(auth:SIMPLE)/600000 is closing > at > org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1368) > at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1340) > at > org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1540) > at > org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1597) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:21367) > at > org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:290) > at > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:147) > at > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:55) > at > org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:174) > at > org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:216) > at > org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:131) > at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:585) > at > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:164) > at > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:102) > at > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:80) > at > org.apache.hadoop.hbase.master.CatalogJanitor.getMergedRegionsAndSplitParents(CatalogJanitor.java:165) > at > org.apache.hadoop.hbase.master.CatalogJanitor.getMergedRegionsAndSplitParents(CatalogJanitor.java:117) > at > org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:219) > at > org.apache.hadoop.hbase.master.CatalogJanitor.chore(CatalogJanitor.java:99) > at org.apache.hadoop.hbase.Chore.run(Chore.java:80) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.io.IOException: Call 31 not added as the > connection > HOSTNAME-TEST/192.168.1.35:60020/ClientService/hadoop(auth:SIMPLE)/600000 > is closing > at > org.apache.hadoop.hbase.ipc.RpcClient$Connection.addCall(RpcClient.java:469) > at > org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1413) > at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1314) > ... 19 more > > So I want to know what is going wrong? > > Extra information: > hbase issue: https://issues.apache.org/jira/browse/HBASE-8790 > environment: CentOS 5.9 x86_64, java version "1.6.0_45", CDH4.3 > The hbase cluster is a fresh start with only one regionserver. > > > ---------------------------- > Best Wishes
