Craig Condit created HBASE-11396:
------------------------------------
Summary: Invalid meta entries can lead to unstartable master
Key: HBASE-11396
URL: https://issues.apache.org/jira/browse/HBASE-11396
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.98.3
Reporter: Craig Condit
Recently I accidentally kill -9'd all regionservers in my cluster (don't ask..
not my finest moment).
This let to some corruption of the meta table, causing the following exception
to be output on the HBase Master during startup, and culminating with the
Master aborting:
{noformat}
java.lang.IllegalArgumentException: Wrong length: 13, expected 8
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:600)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:578)
at
org.apache.hadoop.hbase.HRegionInfo.getServerName(HRegionInfo.java:1059)
at
org.apache.hadoop.hbase.HRegionInfo.getHRegionInfoAndServerName(HRegionInfo.java:987)
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2678)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:465)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:910)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
No tools that I am aware of were able to clean this up. I added a short patch
to catch and log this exception and return null from
HRegionInfo.getServerName(). This allowed for startup of the cluster, and hbase
hbck to repair the damage.
Creating ticket to submit patch in case anyone else finds this useful.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
