[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604537#comment-13604537
]
ramkrishna.s.vasudevan commented on HBASE-8127:
-----------------------------------------------
@Jeffrey
Is there a way to differentiate a deadServer that was really dead when the
master came up and the one that went down before the master could finish
initialization.
Now when the already dead Servers are added to the deadServers list we get into
this problem.
Another thing is even before the patch ie. HBASE-7824 we were ensuring the
region was removed from RIT but the znode was not getting delted in the case
where the znode for Disabled table was in OPENING state. But in a normal case
i don't think this could happen. Need to check this.
I can understand from your patch that it does optimization of the ROOT and META
to be splitted earlier and then the other regions and you depend more on the
SSH to do the split for the userregions. Let me think more and come up with a
patch if possible.
> Region of a disabling or disabled table could be stuck in transition state
> when RS dies during Master initialization
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-8127
> URL: https://issues.apache.org/jira/browse/HBASE-8127
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.5
> Reporter: Jeffrey Zhong
> Assignee: Jeffrey Zhong
> Fix For: 0.94.7
>
> Attachments: hbase-8127_v1.patch, reproduce-hang.patch
>
>
> The issue happens when a RS dies during a master starts up. After the RS
> reports open to the new master instance and dies immediately thereafter, the
> RITs of disabling tables(or disabled table) on the died RS will be in RIT
> state forever.
> I attached a patch to simulate the situation and you can run the following
> command to reproduce the issue:
> {code}mvn test -PlocalTests
> -Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
> Basically, we skip regions of a dead server inside
> AM.processDeadServersAndRecoverLostRegions as the following code and relies
> on SSH to process those skipped regions:
> {code}
> for (Pair<HRegionInfo, Result> deadRegion : deadServer.getValue()) {
> nodes.remove(deadRegion.getFirst().getEncodedName());
> }
> {code}
> While in SSH, we skip regions of disabling(or disabled table) again by
> function processDeadRegion. Finally comes to the issue that RITs of
> disabling(or disabled table) stuck there forever.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
