[
https://issues.apache.org/jira/browse/HBASE-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115828#comment-13115828
]
stack commented on HBASE-4497:
------------------------------
Thanks J-D. Thats what I was too lazy to looksee for myself. Looks like we
are doing enough tickling. Weird that timeout monitor can cut in, region can
be assigned elsewhere AND successfully update meta before this comes back.
Here is from Rams email up on list earlier with log snippets:
{code}
RS1
===
2011-09-23 22:34:34,000 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: addToOnlineRegions is
doneREGION => {NAME =>
't5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.', TableName => 't5',
STARTKEY => '', ENDKEY => '', ENCODED => 2d06b3ca4d398ec96920ae86441a68c9,}
2011-09-23 22:34:34,009 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Updated row t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9. in region
.META.,,1 with serverName=linux76,60020,1316796517682
2011-09-23 22:34:34,009 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Done with post open
deploy taks for region=t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.,
daughter=false
2011-09-23 22:34:34,009 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x1328ceaa1ff0037 Attempting to transition node
2d06b3ca4d398ec96920ae86441a68c9 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-09-23 22:34:34,038 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Completed
the OPEN of region t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9. but
when transitioning from OPENING to OPENED got a version mismatch, someone
else clashed so now unassigning -- closing region
2011-09-23 22:34:34,038 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Closing t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.: disabling
compactions & flushes
2011-09-23 22:34:34,038 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Updates disabled for region
t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.
2011-09-23 22:34:34,038 DEBUG org.apache.hadoop.hbase.regionserver.Store:
closed f5
2011-09-23 22:34:34,038 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.
RS2
===
2011-09-23 22:33:56,546 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x1328ceaa1ff0039 Successfully transitioned node
2d06b3ca4d398ec96920ae86441a68c9 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
2011-09-23 22:33:56,845 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Post open deploy tasks
for region=t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.,
daughter=false
2011-09-23 22:33:56,845 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: addToOnlineRegions is
doneREGION => {NAME =>
't5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.', TableName => 't5',
STARTKEY => '', ENDKEY => '', ENCODED => 2d06b3ca4d398ec96920ae86441a68c9,}
2011-09-23 22:33:56,856 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Updated row t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9. in region
.META.,,1 with serverName=linux146,60020,1316796499216
2011-09-23 22:33:56,856 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Done with post open
deploy taks for region=t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.,
daughter=false
2011-09-23 22:33:58,887 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x1328ceaa1ff0039 Attempting to transition node
2d06b3ca4d398ec96920ae86441a68c9 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-09-23 22:33:58,893 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x1328ceaa1ff0039 Successfully transitioned node
2d06b3ca4d398ec96920ae86441a68c9 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-09-23 22:33:58,893 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Opened
t5,,1316797380065.2d06b3ca4d398ec96920ae86441a68c9.
{code}
> If region opening fails after updating META HBCK reports it as inconsistent
> and scanning the region throws NSRE
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-4497
> URL: https://issues.apache.org/jira/browse/HBASE-4497
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Priority: Critical
>
> As per the discussion in the mail chain "HBCK reporting of possible mismatch
> in RS assignment" this JIRA is created.
> Consider two RS-> RS1 and RS2.
> A region tries to open in RS1. But it takes a while. The RS1 has still not
> updated meta and transitioned the node from OPENING to OPENED
> So timeout assigns the region to RS2. RS2 successfully updates the META and
> opens the region.
> Now RS1 tries to act on the region by first updating the META and then
> transiting the node to OPENING to OPENED.
> RS1 transiting the node to OPENING to OPENED will fail. But the META entry
> will have RS1 as the latest.
> Now HBCK reports this as an inconsistency and if we try to scan the Region we
> get NotServingRegionException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
