Hi Team, We are using hbase 0.98.1 and hoop-2.2.0 in our system. We notice following issue very frequently in our site. Once this issue starts it keeps coming until we don't restart our application server. How do we fix this issue.
) Encountered problems when prefetch hbase:meta table: : org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions: Mon Nov 03 13:14:56 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:14:57 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:14:57 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:14:58 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:15:00 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:15:04 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:15:14 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:15:24 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:15:34 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException Mon Nov 03 13:15:44 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@361f53dd, java.io.IOException: com.google.protobuf.ServiceException: java.lang.NullPointerException at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:129) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:710) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1158) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1222) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1110) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1067) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:908) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:72) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.apache.hadoop.hbase.client.HTable.get(HTable.java:773) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2] at org.datanucleus.store.hbase.HBaseUtils.getResultForObject(HBaseUtils.java:645) [datanucleus-hbase-3.1.1.jar:] at org.datanucleus.store.hbase.HBasePersistenceHandler.fetchObject(HBasePersistenceHandler.java:687) [datanucleus-hbase-3.1.1.jar:] at org.datanucleus.state.JDOStateManager.loadFieldsFromDatastore(JDOStateManager.java:1634) at org.datanucleus.state.JDOStateManager.validate(JDOStateManager.java:3507) at org.datanucleus.ObjectManagerImpl.findObject(ObjectManagerImpl.java:3379) [datanucleus-core-3.1.2.jar:] at org.datanucleus.api.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManager.java:1722) [datanucleus-api-jdo-3.1.2.jar:] at org.datanucleus.api.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManager.java:1740) [datanucleus-api-jdo-3.1.2.jar:] at
