Ted Yu created HBASE-14190:
------------------------------
Summary: Assign hbase:namespace table ahead of user region
assignment
Key: HBASE-14190
URL: https://issues.apache.org/jira/browse/HBASE-14190
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Currently the namespace table region is assigned like user regions.
I spent several hours working with a customer where master couldn't finish
initialization.
Even though master was restarted quite a few times, it went down with the
following:
{code}
2015-08-05 17:16:57,530 FATAL [hdpmaster1:60000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-08-05 17:16:57,530 FATAL [hdpmaster1:60000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.io.IOException: Timedout 300000ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:104)
at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:985)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:779)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
at java.lang.Thread.run(Thread.java:744)
{code}
During previous run(s), namespace table was created, hence leaving an entry in
hbase:meta.
The following if block in TableNamespaceManager#start() was skipped:
{code}
if (!MetaTableAccessor.tableExists(masterServices.getConnection(),
TableName.NAMESPACE_TABLE_NAME)) {
{code}
TableNamespaceManager#start() spins, waiting for namespace region to be
assigned.
There was issue in master assigning user regions.
We tried issuing 'assign' command from hbase shell which didn't work because of
the following check in MasterRpcServices#assignRegion():
{code}
master.checkInitialized();
{code}
This scenario can be avoided if we assign hbase:namespace table after
hbase:meta is assigned but before user table region assignment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
