[ https://issues.apache.org/jira/browse/HBASE-18551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122679#comment-16122679 ]
Umesh Agashe commented on HBASE-18551: -------------------------------------- +1 > [AMv2] UnassignProcedure and crashed regionservers > -------------------------------------------------- > > Key: HBASE-18551 > URL: https://issues.apache.org/jira/browse/HBASE-18551 > Project: HBase > Issue Type: Bug > Components: amv2 > Reporter: stack > Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-18551.master.001.patch, > HBASE-18551.master.002.patch, HBASE-18551.master.003.patch > > > This has been [~uagashe] and my obsession over the last few days, what should > an UnassignProcedure do when it dispatches a CLOSE but the CLOSE fails > because of ConnectException or SocketTimeout. > + We used to let UnassignProcedure continue presuming the Region would be > closed since the server is dead. BUT, if the unassign was part of a > MoveProcedure, the unassign would proceed and the Move would then run WITHOUT > first splitting logs. Bad. > + So, we made it so UnassignProcedure failed; let the upper layers take care > of the failure. See HBASE-18491 that enabled this behavior. BUT, we are since > figuring that even if the UP completes as a failure, since it gives up the > Region lock on completion, another procedure -- say an AssignProcedure -- > could cut in before the ServerCrashProcedure had finished and again there > could be dataloss. > + Now we are thinking the UP should hold on to the Region lock until we are > signalled by a ServerCrashProcedure; only then let go of the region. The UP > has context that is hard to pass another. Waiting on a SCP has the UP living > on for what could be a good amount of time. It might be ok if we can suspend > the procedure. > There is a good sample scenario that came up doing the no-regions-on-master > issue, HBASE-18511. When meta is not on master, TestSplitTransactionOnCluster > is failing. It fails because though the test completes, the tests commonly > kill a RegionServer. The teardown for the test runs before we've noticed the > aborted RS. So, the disable of the table in the teardown prepartory to our > deleting the test table as part of clean up, goes to unassign regions but the > unassign fails against the aborted server. > Good stuff. -- This message was sent by Atlassian JIRA (v6.4.14#64029)