Julian Chandra Sutadi created HBASE-30519:
---------------------------------------------
Summary: RegionReplicaFlushHandler does not detect a dropped table
when the isTableDisabled check fails with TableNotFoundException
Key: HBASE-30519
URL: https://issues.apache.org/jira/browse/HBASE-30519
Project: HBase
Issue Type: Bug
Reporter: Julian Chandra Sutadi
HBASE-29188 reported that region servers abort with {{TableNotFoundException}}
after a table with region replicas is disabled and dropped. The issue was fixed
by adding {{isTableDisabledOrDropped}} to {{{}RegionReplicaFlushHandler{}}}, so
that a failed flush to the primary replica is ignored if the table is disabled
or no longer exists.
However, the catch block checks the flush error instead of the exception thrown
by the {{isTableDisabled}} check. Since the method already returns early when
the flush error is a {{{}TableNotFoundException{}}}, this branch can never be
true:
{code:java}
// RegionReplicaFlushHandler.isTableDisabledOrDropped
private boolean isTableDisabledOrDropped(IOException error, HRegion region) {
if (error instanceof TableNotFoundException) { // error = the flush
failure
return true;
}
try {
if
(FutureUtils.get(connection.getAdmin().isTableDisabled(region.getRegionInfo().getTable())))
{
return true;
}
} catch (IOException e) { // e =
TableNotFoundException once the table is dropped
if (error instanceof TableNotFoundException) { // checks error instead of
e, always false here
return true;
}
...
}
return false;
}
{code}
After a drop, the flush itself fails with {{UnknownRegionException}} (the
region is gone from meta), not {{TableNotFoundException}}, so the first check
does not match either. As a result, {{isTableDisabledOrDropped}} returns false
and the handler treats it as a normal flush failure:
# If there are retries left, it sleeps and tries again. Before the next
attempt, the while loop checks whether the secondary region is still open.
Since disabling the table closed it, the loop exits and the handler finishes
normally, so the bug has no visible effect.
# If it was the last attempt, there is no retry and no next loop check. The
flush error is rethrown and {{handleException}} aborts the region server.
{code:java}
// RegionReplicaFlushHandler.triggerFlushInPrimaryRegion
while (!region.isClosing() && !region.isClosed() && ...) { // a closed
secondary exits here on the next attempt
...
} catch (IOException e) {
if (isTableDisabledOrDropped(e, region)) {
return;
}
if (!counter.shouldRetry()) {
throw e; // last attempt:
rethrown, handleException aborts the RS
}
...
}
}
{code}
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
