Hi,
I use Hbase 1.0 cmf-5.4. Hbase is used for store users' event log. I
write our data with php+thrift. At the beginning everything works
fine. After during a time Thrift server throws below error. Do you
have any idea about reason of error ?
Thanks
2015-11-17 21:15:11,537 WARN
org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler: Failed
after attempts=5, exceptions:
Tue Nov 17 21:14:31 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:14:41 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:14:51 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:15:01 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:15:11 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=5, exceptions:
Tue Nov 17 21:14:31 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:14:41 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:14:51 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:15:01 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
Tue Nov 17 21:15:11 UTC 2015,
RpcRetryingCaller{globalStartTime=1447794871497, pause=100,
retries=5}, java.io.IOException: hconnection-0x73296f2 closed
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:147)
at org.apache.hadoop.hbase.client.HTable.incrementColumnValue(HTable.java:1279)
at org.apache.hadoop.hbase.client.HTable.incrementColumnValue(HTable.java:1222)
at
org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.atomicIncrement(ThriftServerRunner.java:1355)
at
org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.atomicIncrement(ThriftServerRunner.java:1346)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
at com.sun.proxy.$Proxy14.atomicIncrement(Unknown Source)
at
org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$atomicIncrement.getResult(Hbase.java:4438)
at
org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$atomicIncrement.getResult(Hbase.java:4422)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x73296f2 closed
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1683)
at
org.apache.hadoop.hbase.client.ZooKeeperRegistry.isTableOnlineState(ZooKeeperRegistry.java:103)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isTableDisabled(ConnectionManager.java:924)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1093)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1081)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:902)
at
org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
... 18 more
--
Talat UYARER
Websitesi: http://talat.uyarer.com
Twitter: http://twitter.com/talatuyarer
Linkedin: http://tr.linkedin.com/pub/talat-uyarer/10/142/304
