Can you turn on DEBUG logging ? That may give you more idea on why the region doesn't open.
You can also use the http://hbase.apache.org/book.html#hfile_tool to inspect hfiles in this region. Cheers On Wed, Sep 20, 2017 at 5:45 AM, Adam Hunt <[email protected]> wrote: > I'm are running HBase 1.1.8 on MapR 5.2.1 and I have a table with a little > over 1000 regions. All the regions load fine, except for one, which is > stuck in the opening state. I've tried restarting the masters and all the > region servers, but I'm getting the same result. The logs don't seem very > helpful. They show that the request to open the region, but it never > succeeds. > > 2017-09-20 05:26:02,095 INFO > [PriorityRpcServer.handler=3,queue=1,port=16020] > regionserver.RSRpcServices: Open whois-versioned,bobandamy.net > ,1505420182154.489a652d37429afe19955512f644f64f. > 2017-09-20 05:26:02,103 INFO > [StoreOpener-489a652d37429afe19955512f644f64f-1] hfile.CacheConfig: > blockCache=LruBlockCache{blockCount=17212, currentSize=1451765224, > freeSize=4952805912, maxSize=6404571136, heapSize=1451765224, > minSize=6084342272, minFactor=0.95, multiSize=3042171136, multiFactor=0.5, > singleSize=1521085568, singleFactor=0.25}, cacheDataOnRead=true, > cacheDataOnWrite=false, cacheIndexesOnWrite=false, > cacheBloomsOnWrite=false, cacheEvictOnClose=false, > cacheDataCompressed=false, prefetchOnOpen=false > 2017-09-20 05:26:02,103 INFO > [StoreOpener-489a652d37429afe19955512f644f64f-1] > compactions.CompactionConfiguration: size [134217728, > 9223372036854775807); > files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point > 2684354560; major period 604800000, major jitter 0.500000, min locality to > compact 0.000000 > 2017-09-20 05:26:02,106 INFO > [StoreOpener-489a652d37429afe19955512f644f64f-1] hfile.CacheConfig: > blockCache=LruBlockCache{blockCount=17212, currentSize=1451765224, > freeSize=4952805912, maxSize=6404571136, heapSize=1451765224, > minSize=6084342272, minFactor=0.95, multiSize=3042171136, multiFactor=0.5, > singleSize=1521085568, singleFactor=0.25}, cacheDataOnRead=true, > cacheDataOnWrite=false, cacheIndexesOnWrite=false, > cacheBloomsOnWrite=false, cacheEvictOnClose=false, > cacheDataCompressed=false, prefetchOnOpen=false > > hbck reports this error: > > ERROR: Region { meta => > whois-versioned,bobandamy.net,1505420182154.489a652d37429afe19955512f644f6 > 4f., > hdfs => > maprfs:///hbase/data/default/whois-versioned/ > 489a652d37429afe19955512f644f64f, > deployed => , replicaId => 0 } not deployed on any region server. > 2017-09-20 05:29:10,602 INFO [main] util.HBaseFsck: Handling overlap > merges in parallel. set hbasefsck.overlap.merge.parallel to false to run > serially. > ERROR: There is a hole in the region chain between bobandamy.net and > bodyinbalanceshop.com. You need to create a new .regioninfo and region > dir > in hdfs to plug the hole. > ERROR: Found inconsistency in table whois-versioned > > Repairing the table created more of a mess, and didn't fix this issue. I've > tried the assigning it from the shell, but the operation just times out. > Any ideas on how to debug this issue in order to recover the data would be > greatly appreciated. Thank you. >
