[jira] Commented: (HBASE-3381) Interrupt of a region open comes across as a successful open

[Jonathan Gray (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HBASE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12973986#action_12973986
 ] 

Jonathan Gray commented on HBASE-3381:
--------------------------------------

+1 on all but the last change in HBCK.

{noformat}

@@ -397,7 +399,8 @@ public class HBaseFsck {
       return;
     } else if (inMeta && !shouldBeDeployed && !isDeployed) {
       // offline regions shouldn't cause complaints
-      LOG.debug("Region " + descriptiveName + " offline, ignoring.");
+      LOG.debug("Region " + descriptiveName + " offline, splitParent=" + 
splitParent +
+        ", ignoring.");
       return;
     } else if (recentlyModified) {
       LOG.info("Region " + descriptiveName + " was recently modified -- 
skipping");
{noformat}

This version of HBCK changes the {{shouldBeDeployed}} to use the new ZK-based 
table disabling...

{noformat}
    boolean shouldBeDeployed = inMeta && !isTableDisabled(hbi.metaEntry);
{noformat}

and...

{noformat}
  private boolean isTableDisabled(HRegionInfo regionInfo) {
    return disabledTables.contains(regionInfo.getTableDesc().getName());
  }
{noformat}

I think there should be (yet another) boolean in HBCK that is offlineInMeta.  
And then separate checks for the possible good/bad states of that flag.  (and 
this log message should remain as it was since it's not relevant for splits).

> Interrupt of a region open comes across as a successful open
> ------------------------------------------------------------
>
>                 Key: HBASE-3381
>                 URL: https://issues.apache.org/jira/browse/HBASE-3381
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>             Fix For: 0.90.0
>
>         Attachments: 3381.txt
>
>
> Meta was offline when below happened:
> {code}
> 2010-12-21 19:45:23,023 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
> regionserver:60020-0x12d0a53c540000e Attempting to transition node 
> 337038b50e467fbd6b031f278bbd9c22 from RS_ZK_REGION_OPENING to 
> RS_ZK_REGION_OPENING
> 2010-12-21 19:45:23,046 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
> regionserver:60020-0x12d0a53c540000e Successfully transitioned node 
> 337038b50e467fbd6b031f278bbd9c22 from RS_ZK_REGION_OPENING to 
> RS_ZK_REGION_OPENING
> 2010-12-21 19:45:26,379 DEBUG 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Interrupting 
> thread Thread[PostOpenDeployTasks:337038b50e467fbd6b031f278bbd9c22,5,main]
> 2010-12-21 19:45:26,379 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
> regionserver:60020-0x12d0a53c540000e Attempting to transition node 
> 337038b50e467fbd6b031f278bbd9c22 from RS_ZK_REGION_OPENING to 
> RS_ZK_REGION_OPENED
> 2010-12-21 19:45:26,381 WARN 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Exception 
> running postOpenDeployTasks; region=337038b50e467fbd6b031f278bbd9c22
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Interrupted
>     at 
> org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:364)
>     at 
> org.apache.hadoop.hbase.catalog.MetaEditor.updateRegionLocation(MetaEditor.java:146)
>     at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1331)
>     at 
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:195)
> ...
> {code}
> So, we timed out trying to open the region but rather than close the region 
> because edit failed, we missed seeing the InterruptedException.
> Here is suggested fix:
> {code}
> diff --git a/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 
> b/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
> index 7bf680d..2b0078c 100644
> --- a/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
> +++ b/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
> @@ -339,7 +339,7 @@ public class MetaReader {
>      get.addFamily(HConstants.CATALOG_FAMILY);
>      byte [] meta = getCatalogRegionNameForRegion(regionName);
>      Result r = catalogTracker.waitForMetaServerConnectionDefault().get(meta, 
> get);
> -    if(r == null || r.isEmpty()) {
> +    if (r == null || r.isEmpty()) {
>        return null;
>      }
>      return metaRowToRegionPair(r);
> {code}
> Let me try it.
> W/o this, what we see is hbck showing that region is on server X but in 
> .META. it shows as being on Y (its pre-balance server)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.