[
https://issues.apache.org/jira/browse/HBASE-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11396:
---------------------------
Resolution: Fixed
Fix Version/s: 0.98.4
0.99.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Integrated to 0.98 and trunk.
Thanks for the patch, Craig.
> Invalid meta entries can lead to unstartable master
> ---------------------------------------------------
>
> Key: HBASE-11396
> URL: https://issues.apache.org/jira/browse/HBASE-11396
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.98.3
> Reporter: Craig Condit
> Fix For: 0.99.0, 0.98.4
>
> Attachments: HBASE-11396-v1.patch
>
>
> Recently I accidentally kill -9'd all regionservers in my cluster (don't
> ask.. not my finest moment).
> This let to some corruption of the meta table, causing the following
> exception to be output on the HBase Master during startup, and culminating
> with the Master aborting:
> {noformat}
> java.lang.IllegalArgumentException: Wrong length: 13, expected 8
> at
> org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:600)
> at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:578)
> at
> org.apache.hadoop.hbase.HRegionInfo.getServerName(HRegionInfo.java:1059)
> at
> org.apache.hadoop.hbase.HRegionInfo.getHRegionInfoAndServerName(HRegionInfo.java:987)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2678)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:465)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:910)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
> No tools that I am aware of were able to clean this up. I added a short patch
> to catch and log this exception and return null from
> HRegionInfo.getServerName(). This allowed for startup of the cluster, and
> hbase hbck to repair the damage.
> Creating ticket to submit patch in case anyone else finds this useful.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
