[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586855#comment-13586855
]
rajeshbabu commented on HBASE-6469:
-----------------------------------
@Lars,
bq.If a background can continue the transaction, why can't another (user
initiated) attempt to either enable or disable the table?
Multiple calls of enable/disable may cause inconsistencies.
Here is the comment from EnableTableHandler/DisableTableHandler constructor.
{code}
// There could be multiple client requests trying to disable or enable
// the table at the same time. Ensure only the first request is honored
// After that, no other requests can be accepted until the table reaches
// DISABLED or ENABLED.
{code}
bq.The expectation here is that a user won't care what care what state the
table is currently in. If a disable is triggered the result should be a
disabled table (and similar for an enable).
I agree but user need to know about the current state of
assignments/unassignments(as of now) before calling force enable/disable.
bq.Can we just make so that we allow transitioning from ENABLING to both
ENABLED or DISABLED and from DISABLING to both ENABLED or DISABLED?
If we transition from ENABLING to DISABLED then the regions which are in
transtion or yet to start assignment wont be assigned. Some times even assigned
regions will be unassigned.
Below are the some code snippets which are used to handle inconsistencies with
disable and balance race.
{code}
private void assign(RegionState state,
final boolean setOfflineInZK, final boolean forceNewPlan) {
....
if (isDisabledorDisablingRegionInRIT(region)) {
return;
}
...
}
{code}
After completion of rit transition we will check once more that table is
disabling or disabled
{code}
boolean disabled = getZKTable().isDisablingOrDisabledTable(
regionInfo.getTableNameAsString());
if (!serverManager.isServerOnline(serverName) && !disabled) {
LOG.info("Opened region " + regionNameStr
+ "but the region server is offline, reassign the region");
assign(regionInfo, true);
} else if (disabled) {
// if server is offline, no hurt to unassign again
LOG.info("Opened region " + regionNameStr
+ "but this table is disabled, triggering close of region");
unassign(regionInfo);
}
{code}
During master restart we will skip regions assignment of disabling/disabled
tables(This case is if master restarted after transition).
{code}
Set<String> disabledOrDisablingOrEnabling =
ZKTable.getDisabledOrDisablingTables(watcher);
disabledOrDisablingOrEnabling.addAll(ZKTable.getEnablingTables(watcher));
// Scan META for all user regions, skipping any disabled tables
Map<HRegionInfo, ServerName> allRegions = MetaReader.fullScan(
catalogTracker, disabledOrDisablingOrEnabling, true);
{code}
Exactly reverse(assign instead of unassign) will happen if we transition from
DISABLING to ENABLED.
code snippet from ClosedRegionHandler.process()
{code}
if (this.assignmentManager.getZKTable().
isDisablingOrDisabledTable(this.regionInfo.getTableNameAsString())) {
assignmentManager.offlineDisabledRegion(regionInfo);
return;
}
// ZK Node is in CLOSED state, assign it.
assignmentManager.getRegionStates().updateRegionState(
regionInfo, RegionState.State.CLOSED, null);
// This below has to do w/ online enable/disable of a table
assignmentManager.removeClosedRegion(regionInfo);
assignmentManager.assign(regionInfo, true);
{code}
Please correct me if I am wrong. Thanks Lars.
> Failure on enable/disable table will cause table state in zk to be left as
> enabling/disabling until master is restarted
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6469
> URL: https://issues.apache.org/jira/browse/HBASE-6469
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.2, 0.96.0
> Reporter: Enis Soztutar
> Assignee: rajeshbabu
> Fix For: 0.96.0, 0.94.7
>
> Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch,
> HBASE-6469_3.patch, HBASE-6469.patch
>
>
> In Enable/DisableTableHandler code, if something goes wrong in handling, the
> table state in zk is left as ENABLING / DISABLING. After that we cannot force
> any more action from the API or CLI, and the only recovery path is restarting
> the master.
> {code}
> if (done) {
> // Flip the table to enabled.
> this.assignmentManager.getZKTable().setEnabledTable(
> this.tableNameStr);
> LOG.info("Table '" + this.tableNameStr
> + "' was successfully enabled. Status: done=" + done);
> } else {
> LOG.warn("Table '" + this.tableNameStr
> + "' wasn't successfully enabled. Status: done=" + done);
> }
> {code}
> Here, if done is false, the table state is not changed. There is also no way
> to set skipTableStateCheck from cli / api.
> We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
