Normally this path: /hbase/data/default/TestTable/TestTable/ d5b64ad96deb3b47467db97669c009fb should be /hbase/data/default/TestTable/d5b64ad96deb3b47467db97669c009fb.
Meaning that the table descriptor somehow was written under the wrong folder (there should be only one TestTable in the path). The masters logs might reveal something at the time of the table creation. Enis On Sun, Nov 22, 2015 at 6:02 PM, Pankaj kr <[email protected]> wrote: > Thanks Ted, > > All regions are in the same condition. > Even the table descriptor also, > > drwxr-xr-x - hbase hadoop 0 2015-11-18 00:04 > /hbase/data/default/TestTable/TestTable/.tabledesc > -rw-r--r-- 3 hbase hadoop 289 2015-11-18 00:04 > /hbase/data/default/TestTable/TestTable/.tabledesc/.tableinfo.0000000001 > > So restarting HM/RS doesn't help. > > Regards, > Pankaj > > -----Original Message----- > From: Ted Yu [mailto:[email protected]] > Sent: 21 November 2015 22:24 > To: [email protected] > Subject: Re: Table regions are written in wrong path in HDFS > > Can you trace this region through master / region server log to see if > there is some clue ? > > Cheers > > > On Nov 21, 2015, at 2:56 AM, Pankaj kr <[email protected]> wrote: > > > > Hi Folks, > > > > We met a very weird scenario. > > We are running PE tool, during testing we found all regions are in > transition in state FAILED_OPEN. > > > > Region server are failed to open the regions with below exception, > > 2015-11-18 02:20:38,503 | ERROR | RS_OPEN_REGION-HOSTNAME:PORT-2 | > Failed open of > region=TestTable,00000000000000000025000000,1447776261671.d5b64ad96deb3b47467db97669c009fb., > starting to roll back the global memstore size. | > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:386) > > java.lang.IllegalStateException: Could not instantiate a region instance. > > at > org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:6229) > > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6536) > > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6508) > > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6464) > > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6415) > > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363) > > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129) > > at > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > Caused by: java.lang.reflect.InvocationTargetException > > at > sun.reflect.GeneratedConstructorAccessor17.newInstance(Unknown Source) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at > java.lang.reflect.Constructor.newInstance(Constructor.java:422) > > at > org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:6226) > > ... 10 more > > Caused by: java.lang.IllegalArgumentException: Need table descriptor > > at > org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:695) > > at > org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:672) > > ... 14 more > > > > In HDFS, Table region was written as below, > > drwxr-xr-x - hbase hadoop 0 2015-11-18 00:05 > /hbase/data/default/TestTable/TestTable/d5b64ad96deb3b47467db97669c009fb > > -rw-r--r-- 3 hbase hadoop 96 2015-11-18 00:05 > /hbase/data/default/TestTable/TestTable/d5b64ad96deb3b47467db97669c009fb/.regioninfo > > > > Here table name "TestTable" has come twice, how is that possible? > > > > Have anyone met this scenario? > > We are using HBase 1.0.2 and HDFS 2.7.2 version. > > > > Any suggestion/help would be much appreciated. > > > > Regards, > > Pankaj >
