[
https://issues.apache.org/jira/browse/HBASE-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16366272#comment-16366272
]
stack commented on HBASE-19851:
-------------------------------
I saw another version of this issue in nightly. We have reached one of the
states below:
{code}
case MERGE_TABLE_REGIONS_POST_OPERATION:
case MERGE_TABLE_REGIONS_OPEN_MERGED_REGION:
case MERGE_TABLE_REGIONS_POST_MERGE_COMMIT_OPERATION:
case MERGE_TABLE_REGIONS_UPDATE_META:
String msg = this + " We are in the " + state + " state."
+ " It is complicated to rollback the merge operation that region
server is working on."
+ " Rollback is not supported and we should let the merge operation
to complete";
LOG.warn(msg);
// PONR
throw new UnsupportedOperationException(this + " unhandled state=" +
state);
{code}
We throw the UnsupportedOperationException which then tickles a CODE-BUG
warning in the framework which is its way of saying not-supposed-to-happen and
it is right; the merge procedure is not complete.
{code}
2018-02-15 16:18:58,492 WARN [PEWorker-1]
assignment.MergeTableRegionsProcedure(311): Failed rollback attempt step
MERGE_TABLE_REGIONS_UPDATE_META for merging the regions
[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748] in table
testRollbackAndDoubleExecution
java.lang.UnsupportedOperationException: pid=46,
state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
abort requested; MergeTableRegionsProcedure
table=testRollbackAndDoubleExecution,
regions=[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748],
forcibly=false unhandled state=MERGE_TABLE_REGIONS_UPDATE_META
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:291)
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:78)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:199)
at org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:859)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1356)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1312)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1181)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
2018-02-15 16:18:58,497 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
CODE-BUG: Uncaught runtime exception for pid=46,
state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
abort requested; MergeTableRegionsProcedure
table=testRollbackAndDoubleExecution,
regions=[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748],
forcibly=false
java.lang.UnsupportedOperationException: pid=46,
state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META,
exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via
MergeTableRegionsProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException:
abort requested; MergeTableRegionsProcedure
table=testRollbackAndDoubleExecution,
regions=[dcd17e940fbb453cf6df1cee87b0cec7, 2b03bbfd418c9b3533bc2bb3cf6ce748],
forcibly=false unhandled state=MERGE_TABLE_REGIONS_UPDATE_META
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:291)
at
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.rollbackState(MergeTableRegionsProcedure.java:78)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:199)
at org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:859)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1356)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1312)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1181)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
{code}
> ProcV2 framework's rollback() needs to maintain stack like execute()
> --------------------------------------------------------------------
>
> Key: HBASE-19851
> URL: https://issues.apache.org/jira/browse/HBASE-19851
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
> Affects Versions: 2.0.0-beta-1
> Reporter: Umesh Agashe
> Priority: Major
> Fix For: 2.0.0
>
>
> Refer to HBASE-19839 and detailed description there. Bug showed up as a flaky
> test TestMergeTableRegionsProcedure.testRollbackAndDoubleExecution(). In
> short, when rollback() is called after step 5
> (MERGE_TABLE_REGIONS_CLOSE_REGIONS), rollback of the procedure tries to
> reassign two regions that are to be merged. It spawns 2 AssignProcedures and
> don't wait on them to finish. Rollback should have ability to have
> sub-procedures and separate proc stack.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
