The .META. table is not flushed. But then, I killed -9 all RS and master, then re-start hbase. Everything become ok.
On Tue, Aug 25, 2009 at 12:35 PM, Ryan Rawson <[email protected]> wrote: > Did you flush .META. table as well? > > -ryan > > On Mon, Aug 24, 2009 at 9:26 PM, Schubert Zhang<[email protected]> wrote: > > -1 > > > > I tested reliability of RegionServer crash, and found following problem: > > > > Test: > > (1) A table, inserted millions of rows. the I use hbase shell commant to > > flush a memstores to disk, there are 25 regions. > > (2) Kill a RS (no ROOT and META region), then scan the whole table, > success. > > (3) Kill another RS (with META region), then scan the whole table, fail. > > Maybe meta and data mismatch. > > (strangely there are 28 regions now, why? and following is the err > log!) > > > > > > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to > > contact region server 10.24.1.16:60020 for region > > TestTable,0001698618,1251172170797, row '0001698618', but failed after > > 10 attempts. > > Exceptions: > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > > > at > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:937) > > at > org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1878) > > at > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1984) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$ScanTest.testRow(PerformanceEvaluation.java:716) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:588) > > at > org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:815) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:369) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:345) > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:518) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) > > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to > > contact region server 10.24.1.16:60020 for region > > TestTable,0001698618,1251172170797, row '0001698618', but failed after > > 10 attempts. > > Exceptions: > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > > > at > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:937) > > at > org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1878) > > at > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1984) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$ScanTest.testRow(PerformanceEvaluation.java:716) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:588) > > at > org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:815) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:369) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:345) > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:518) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) > > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to > > contact region server 10.24.1.16:60020 for region > > TestTable,0001698618,1251172170797, row '0001698618', but failed after > > 10 attempts. > > Exceptions: > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > org.apache.hadoop.hbase.NotServingRegionException: > > org.apache.hadoop.hbase.NotServingRegionException: > > TestTable,0001698618,1251172170797 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2261) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1877) > > at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > > > at > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:937) > > at > org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1878) > > at > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1984) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$ScanTest.testRow(PerformanceEvaluation.java:716) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:588) > > at > org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:815) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:369) > > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:345) > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:518) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) > > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > > > > > > > > > > > > On Wed, Aug 19, 2009 at 3:39 AM, Schubert Zhang <[email protected]> > wrote: > > > >> +1 > >> > >> I have done our test, it is fine to release. > >> > >> On Wed, Aug 19, 2009 at 1:13 AM, stack <[email protected]> wrote: > >> > >>> Update your binaries. Take down the cluster en masse and update all or > do > >>> it via a rolling restart: > >>> http://wiki.apache.org/hadoop/Hbase/RollingRestart > >>> . > >>> > >>> Regards whats been fixed since RC1, see http://su.pr/1QTH7L. > >>> > >>> St.Ack > >>> > >>> On Tue, Aug 18, 2009 at 12:04 AM, Rocks <[email protected]> > wrote: > >>> > >>> > Now I use candidate-1 version, To fix these bugs, how and what should > I > >>> do > >>> > ? > >>> > > >>> > On Tue, Aug 18, 2009 at 2:20 PM, Ryan Rawson <[email protected]> > >>> wrote: > >>> > > >>> > > A few bugs were fixed, seemingly minor but critical. > >>> > > > >>> > > enjoy! > >>> > > -ryan > >>> > > > >>> > > On Mon, Aug 17, 2009 at 11:16 PM, Rocks<[email protected]> > >>> wrote: > >>> > > > What is the difference between the two candidate version? > >>> > > > > >>> > > > On Tue, Aug 18, 2009 at 1:52 PM, stack <[email protected]> wrote: > >>> > > > > >>> > > >> The second hbase 0.20.0 release candidate is available for > >>> download: > >>> > > >> > >>> > > >> http://people.apache.org/~stack/hbase-0.20.0-candidate-2/< > >>> http://people.apache.org/%7Estack/hbase-0.20.0-candidate-2/<http://people.apache.org/~stack/hbase-0.20.0-candidate-2/> > <http://people.apache.org/~stack/hbase-0.20.0-candidate-2/> > >>> > > >>> > > >> > >>> > > >> Near to 450 issues have been addressed since the 0.19 branch. > >>> Release > >>> > > >> notes > >>> > > >> are available here: http://su.pr/18zcEO < > http://tinyurl.com/8xmyx9 > >>> >. > >>> > > >> > >>> > > >> HBase 0.20.0 runs on Hadoop 0.20.0. Alot has changed since > 0.19.x > >>> > > >> including > >>> > > >> configuration fundamentals. Be sure to read the 'Getting > Started' > >>> > > >> documentation in 0.20.0 available here: http://su.pr/8YQjHO > >>> > > >> > >>> > > >> If you wish to bring your 0.19.x hbase data forward to 0.20.0, > you > >>> > will > >>> > > >> need > >>> > > >> to run a migration. See > >>> > > http://wiki.apache.org/hadoop/Hbase/HowToMigrate. > >>> > > >> First read the overview and then go to the section, 'From 0.19.x > to > >>> > > >> 0.20.x'. > >>> > > >> > >>> > > >> Should we release this candidate as hbase 0.20.0? Please vote > >>> +1/-1 > >>> > by > >>> > > >> Wednesday, August 25th. > >>> > > >> > >>> > > >> Yours, > >>> > > >> The HBasistas > >>> > > >> > >>> > > > > >>> > > > >>> > > >>> > >> > >> > > >
