Sorry Stack. I did type mistake. I meant I am using CHD 5.4.7 and its HBase(1.0.0-cdh5.4.7).
Actually I can not find any meaningful clue. This link[1] has log from the beginning. This exception is threw by thrift. Zk and thrift logging level is DEBUG. [1] http://dpaste.com/1XRW8P9 2015-11-30 22:48 GMT-08:00 Stack <st...@duboce.net>: > On Mon, Nov 30, 2015 at 4:34 PM, Talat Uyarer <ta...@uyarer.com> wrote: > >> Hi, >> >> I use Hbase 1.0 cmf-5.4. > > > Where does this hbase come from Talat? > > > >> Hbase is used for store users' event log. I >> write our data with php+thrift. At the beginning everything works >> fine. After during a time Thrift server throws below error. Do you >> have any idea about reason of error ? >> >> > Anything earlier in the log saying why the connection is getting closed? > > St.Ack > > > >> Thanks >> >> 2015-11-17 21:15:11,537 WARN >> org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler: Failed >> after attempts=5, exceptions: >> Tue Nov 17 21:14:31 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:14:41 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:14:51 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:15:01 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:15:11 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after >> attempts=5, exceptions: >> Tue Nov 17 21:14:31 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:14:41 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:14:51 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:15:01 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> Tue Nov 17 21:15:11 UTC 2015, >> RpcRetryingCaller{globalStartTime=1447794871497, pause=100, >> retries=5}, java.io.IOException: hconnection-0x73296f2 closed >> >> at >> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:147) >> at >> org.apache.hadoop.hbase.client.HTable.incrementColumnValue(HTable.java:1279) >> at >> org.apache.hadoop.hbase.client.HTable.incrementColumnValue(HTable.java:1222) >> at >> org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.atomicIncrement(ThriftServerRunner.java:1355) >> at >> org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.atomicIncrement(ThriftServerRunner.java:1346) >> at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67) >> at com.sun.proxy.$Proxy14.atomicIncrement(Unknown Source) >> at >> org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$atomicIncrement.getResult(Hbase.java:4438) >> at >> org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$atomicIncrement.getResult(Hbase.java:4422) >> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) >> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) >> at >> org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289) >> at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.io.IOException: hconnection-0x73296f2 closed >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1683) >> at >> org.apache.hadoop.hbase.client.ZooKeeperRegistry.isTableOnlineState(ZooKeeperRegistry.java:103) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isTableDisabled(ConnectionManager.java:924) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1093) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1081) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:902) >> at >> org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78) >> at >> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124) >> ... 18 more >> >> -- >> Talat UYARER >> Websitesi: http://talat.uyarer.com >> Twitter: http://twitter.com/talatuyarer >> Linkedin: http://tr.linkedin.com/pub/talat-uyarer/10/142/304 >> -- Talat UYARER Websitesi: http://talat.uyarer.com Twitter: http://twitter.com/talatuyarer Linkedin: http://tr.linkedin.com/pub/talat-uyarer/10/142/304
