[
https://issues.apache.org/jira/browse/HBASE-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042712#comment-14042712
]
stack commented on HBASE-11396:
-------------------------------
Yeah, was going to say add a test since easy to do but this is good as is I'd
say. Funny we've not had this more often... (Others having 'finest moments' --
smile). Good on you Craig.
> Invalid meta entries can lead to unstartable master
> ---------------------------------------------------
>
> Key: HBASE-11396
> URL: https://issues.apache.org/jira/browse/HBASE-11396
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.98.3
> Reporter: Craig Condit
> Attachments: HBASE-11396-v1.patch
>
>
> Recently I accidentally kill -9'd all regionservers in my cluster (don't
> ask.. not my finest moment).
> This let to some corruption of the meta table, causing the following
> exception to be output on the HBase Master during startup, and culminating
> with the Master aborting:
> {noformat}
> java.lang.IllegalArgumentException: Wrong length: 13, expected 8
> at
> org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:600)
> at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:578)
> at
> org.apache.hadoop.hbase.HRegionInfo.getServerName(HRegionInfo.java:1059)
> at
> org.apache.hadoop.hbase.HRegionInfo.getHRegionInfoAndServerName(HRegionInfo.java:987)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2678)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:465)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:910)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
> No tools that I am aware of were able to clean this up. I added a short patch
> to catch and log this exception and return null from
> HRegionInfo.getServerName(). This allowed for startup of the cluster, and
> hbase hbck to repair the damage.
> Creating ticket to submit patch in case anyone else finds this useful.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
