See my 2nd mail regarding this: Didn't help. We're still heavily investigating this, so far without any progress.
On Nov 7, 2012, at 5:43 PM, Leonid Fedotov wrote: > The first to try is > > hbase hbck -repair > > Check here: > http://hbase.apache.org/book/apbs04.html#d2067e11173 > > Thank you! > > Sincerely, > Leonid Fedotov > > > On Nov 6, 2012, at 10:08 PM, Mesika, Asaf wrote: > >> After restart, running hbck results in: >> 12/11/07 06:07:28 DEBUG client.MetaScanner: Scanning .META. starting at row= >> for max=2147483647 rows using >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@51b4a74b >> >> 12/11/07 06:07:28 INFO util.HBaseFsck: getHTableDescriptors == tableNames => >> [] >> Exception in thread "main" java.lang.NullPointerException >> at >> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:560) >> at >> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:346) >> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382) >> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120) >> >> HBase is version 0.94.0 >> >> >> On Nov 7, 2012, at 7:28 AM, Mesika, Asaf wrote: >> >>> Hi, >>> >>> I have a cluster with 10 RS and 1 Master. >>> From some reason, any call to the HBase fails. I ran hbck and got the same >>> failure: >>> >>> Tue Nov 06 13:27:07 UTC 2012, >>> org.apache.hadoop.hbase.client.ScannerCallable@7d8a8ce2, >>> org.apache.hadoop.hbase.NotServingRegionException: >>> org.apache.hadoop.hbase.NotServingRegionException: Region is not online: >>> .META.,,1 >>> >>> at >>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:196) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:48) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:126) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:123) >>> at >>> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:359) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:123) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:99) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:77) >>> at >>> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:61) >>> at >>> org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntries(HBaseFsck.java:2234) >>> at >>> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:334) >>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382) >>> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120) >>> >>> >>> Anyone has any idea how to fix it without the obvious restart? >>> >> >
