Hi Dylan, $HBASE_HOME/bin/hbase hbck -fix
the above command can bring back the cluster to normal state. Just if master restarted while ROOT/META region in closing(during move or balance) then the problem you have reported will come easily. Thanks for detailed logs. Raised an issue for this. You can follow HBASE-8353. <https://issues.apache.org/jira/browse/HBASE-8353> On Tue, Apr 16, 2013 at 7:24 AM, Ted Yu <[email protected]> wrote: > I think this question would be more appropriate for HBase user mailing > list. > > Moving hadoop user to bcc. > > Please tell us the HBase version you are using. > > Thanks > > On Mon, Apr 15, 2013 at 6:51 PM, dylan <[email protected]> wrote: > > > Hi**** > > > > ** ** > > > > I am a newer for hadoop, and set up hadoop with tarball . I have 5 nodes > > for cluster, 2 NN nodes with QJM (3 Journal Nodes, one of them on DN > node. > > ), 3 DN nodes with zookeepers, It works fine. When I reboot one data > > node machine which includes zookeeper, after that , restart all > processes. > > The hadoop works fine, but hbase not. I cannot disable tables and drop > > tables.**** > > > > ** ** > > > > The logs an follows:**** > > > > The Hbase HMaster log:**** > > > > DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Attempted to > > unassign region -ROOT-,,0.70236052 but it is not currently assigned > anywhere > > **** > > > > ,683 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in > > transition timed out: -ROOT-,,0.70236052 state=CLOSING, > ts=1366001558865, > > server=Master,60000,1366001238313**** > > > > ,683 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has > > been CLOSING for too long, this should eventually complete or the server > > will expire, send RPC again**** > > > > 10,684 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting > > unassignment of region -ROOT-,,0.70236052 (offlining)**** > > > > ** ** > > > > The Hbase HRegionServer log:**** > > > > ** ** > > > > DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Stats: total=7.44 > > MB, free=898.81 MB, max=906.24 MB, blocks=0, accesses=0, hits=0, > > hitRatio=0, cachingAccesses=0, cachingHits=0, cachingHitsRatio=0, > > evictions=0, evicted=0, evictedPerRun=NaN**** > > > > ** ** > > > > The Hbase Web show：**** > > > > Region State**** > > > > 70236052 -ROOT-,,0.70236052 state=CLOSING, ts=Mon Apr 15 12:52:38 CST > > 2013 (75440s ago), server=Master,60000,1366001238313**** > > > > ** ** > > > > How fix it?**** > > > > ** ** > > > > Thanks.**** > > >
