[jira] [Updated] (HBASE-26391) Unexpected crash in the process of deleting a table causes stale data in ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
May updated HBASE-26391:
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
4. HMaster1 crashes before deleting the znode: /hbase/table-lock/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1341),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1330),
org.apache.hadoop.hbase.master.TableLockManager$ZKTableLockManager.tableDeleted(TableLockManager.java:442),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.markTableAsDeleted(MasterProcedureScheduler.java:895),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.completionCleanup(MasterProcedureScheduler.java:261),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.procedureFinished(ProcedureExecutor.java:1415),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:974),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
5. HMaster2 becomes active
6. The table {{mytable}} was deleted, but the znode "/hbase/table-lock/mytable"
was left on ZooKeeper.
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
4.
[jira] [Updated] (HBASE-26391) Unexpected crash in the process of deleting a table causes stale data in ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
May updated HBASE-26391:
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
4. HMaster1 crashes before deleting the znode: /hbase/table-lock/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1341),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1330),
org.apache.hadoop.hbase.master.TableLockManager$ZKTableLockManager.tableDeleted(TableLockManager.java:442),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.markTableAsDeleted(MasterProcedureScheduler.java:895),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.completionCleanup(MasterProcedureScheduler.java:261),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.procedureFinished(ProcedureExecutor.java:1415),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:974),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
5. HMaster2 becomes active
6. The table {{mytable}} was deleted, but the znode "/hbase/table-lock/mytable"
was left on ZooKeeper.
7. Client tries to create {{mytable}} again, but gets {{TableExistsException}}.
However, the table has been created.
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
[jira] [Updated] (HBASE-26391) Unexpected crash in the process of deleting a table causes stale data in ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
May updated HBASE-26391:
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
4. HMaster1 crashes before deleting the znode: /hbase/table-lock/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1341),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1330),
org.apache.hadoop.hbase.master.TableLockManager$ZKTableLockManager.tableDeleted(TableLockManager.java:442),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.markTableAsDeleted(MasterProcedureScheduler.java:895),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.completionCleanup(MasterProcedureScheduler.java:261),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.procedureFinished(ProcedureExecutor.java:1415),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:974),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
5. HMaster2 becomes active
6. The table {{mytable}} was deleted, but the znode "/hbase/table-lock/mytable"
was left on ZooKeeper.
7. Client tries to create {{mytable}} again, but fails due to
{{TableExistsException}}
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
[jira] [Updated] (HBASE-26391) Unexpected crash in the process of deleting a table causes stale data in ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
anonymous updated HBASE-26391:
--
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
4. HMaster1 crashes before deleting the znode: /hbase/table-lock/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1341),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1330),
org.apache.hadoop.hbase.master.TableLockManager$ZKTableLockManager.tableDeleted(TableLockManager.java:442),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.markTableAsDeleted(MasterProcedureScheduler.java:895),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.completionCleanup(MasterProcedureScheduler.java:261),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.procedureFinished(ProcedureExecutor.java:1415),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:974),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
5. HMaster2 becomes active
6. The table {{mytablle}} was deleted, but the znode
"/hbase/table-lock/mytable" was left on ZooKeeper
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 tries to delete znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
[jira] [Updated] (HBASE-26391) Unexpected crash in the process of deleting a table causes stale data in ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
anonymous updated HBASE-26391:
--
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
4. HMaster1 crashes before deleting the znode: /hbase/table-lock/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1341),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1330),
org.apache.hadoop.hbase.master.TableLockManager$ZKTableLockManager.tableDeleted(TableLockManager.java:442),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.markTableAsDeleted(MasterProcedureScheduler.java:895),
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.completionCleanup(MasterProcedureScheduler.java:261),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.procedureFinished(ProcedureExecutor.java:1415),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:974),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
5. HMaster2 becomes active
6. The table {{mytable}} was deleted, but the znode "/hbase/table-lock/mytable"
was left on ZooKeeper
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client request to delete the table {{mytable}};
3. HMaster1 deletes znode in ZooKeeper: /hbase/table/mytable
{code:java}
[org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
{code}
[jira] [Updated] (HBASE-26391) Unexpected crash in the process of deleting a table causes stale data in ZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
anonymous updated HBASE-26391:
--
Summary: Unexpected crash in the process of deleting a table causes stale
data in ZooKeeper (was: Unexpected crash in the process of deleting a table
causes stale data)
> Unexpected crash in the process of deleting a table causes stale data in
> ZooKeeper
> --
>
> Key: HBASE-26391
> URL: https://issues.apache.org/jira/browse/HBASE-26391
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.1
>Reporter: anonymous
>Priority: Major
>
> There is a table {{mytable}} in a HBase cluster with two HMasters and two
> RegionServers.
> 1. Current active HMaster is HMaster1;
> 2. Client request to delete the table {{mytable}};
> 3. HMaster1 tries to delete znode in ZooKeeper: /hbase/table/mytable
> {code:java}
> [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
>
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1367),
>
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeFailSilent(ZKUtil.java:1359),
>
> org.apache.hadoop.hbase.zookeeper.ZKTableStateManager.setDeletedTable(ZKTableStateManager.java:244),
>
> org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.deleteAssignmentState(DeleteTableProcedure.java:383),
>
> org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:136),
>
> org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:57),
>
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139),
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:499),
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1166),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:954),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
> {code}
> 4. HMaster1 crashes before deleting the znode: /hbase/table-lock/mytable
> {code:java}
> [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:238),
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1341),
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1330),
> org.apache.hadoop.hbase.master.TableLockManager$ZKTableLockManager.tableDeleted(TableLockManager.java:442),
>
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.markTableAsDeleted(MasterProcedureScheduler.java:895),
>
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.completionCleanup(MasterProcedureScheduler.java:261),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.procedureFinished(ProcedureExecutor.java:1415),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:974),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:907),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:481),
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java)]
> {code}
> 5. HMaster2 becomes active
> 6. The table {{mytablle}} was deleted, but the znode
> "/hbase/table-lock/mytable" was left on ZooKeeper
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
