[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612421#comment-13612421
]
rajeshbabu commented on HBASE-8127:
-----------------------------------
Thanks for review Jeffrey.
If a RS went down after rebuildUserRegions it wont be added to deadServers map
retruring in that method.
Since its not in deadServers map we are not removing the region entry from
nodes to be processed which will be handled in processRegionInTransition.
{code}
if (!nodes.isEmpty()) {
for (String encodedRegionName : nodes) {
processRegionInTransition(encodedRegionName, null, deadServers);
}
}
{code}
So its not a problem and RIT wont stuck forever.
bq. Why don't you handle this situation in SSH as well by removing RIT null
check?
If we handle this case in SSH without null check then SSH processing will take
more time if any regions of DISABLING/DISABLED tables opened when it was
online. Because we need to contact zookeeper every time even there may not be
znodes of RIT present.
> Region of a disabling or disabled table could be stuck in transition state
> when RS dies during Master initialization
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-8127
> URL: https://issues.apache.org/jira/browse/HBASE-8127
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.5
> Reporter: Jeffrey Zhong
> Assignee: rajeshbabu
> Fix For: 0.94.7
>
> Attachments: HBASE-8127_94_2.patch, HBASE-8127_feedback.patch,
> HBASE-8127.patch, hbase-8127_v1.patch, reproduce-hang.patch
>
>
> The issue happens when a RS dies during a master starts up. After the RS
> reports open to the new master instance and dies immediately thereafter, the
> RITs of disabling tables(or disabled table) on the died RS will be in RIT
> state forever.
> I attached a patch to simulate the situation and you can run the following
> command to reproduce the issue:
> {code}mvn test -PlocalTests
> -Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
> Basically, we skip regions of a dead server inside
> AM.processDeadServersAndRecoverLostRegions as the following code and relies
> on SSH to process those skipped regions:
> {code}
> for (Pair<HRegionInfo, Result> deadRegion : deadServer.getValue()) {
> nodes.remove(deadRegion.getFirst().getEncodedName());
> }
> {code}
> While in SSH, we skip regions of disabling(or disabled table) again by
> function processDeadRegion. Finally comes to the issue that RITs of
> disabling(or disabled table) stuck there forever.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
