[
https://issues.apache.org/jira/browse/HBASE-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971997#action_12971997
]
HBase Review Board commented on HBASE-3362:
-------------------------------------------
Message from: "Jonathan Gray" <[email protected]>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1298/#review2083
-----------------------------------------------------------
Ship it!
a few small comments. i think the loop should change as described in my
comment (busy loop w/ call to currentTimeMillis as i read it). otherwise +1,
good stuff. we need some tickle util class soon :)
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
<http://review.cloudera.org/r/1298/#comment6529>
"on this server" should probably be left in comment to be clear what this
is checking
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
<http://review.cloudera.org/r/1298/#comment6530>
We were not previously but we should probably log this condition
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
<http://review.cloudera.org/r/1298/#comment6531>
This is a busy wait loop?
Should we add a wait/notify on something passed to the thread and w/ a
timeout of the period?
And then we should probably also have some kind of max timeout. Even if
minutes, there could be weird cluster state where the RS misses META
availability but someone else might handle it properly, so max timeout might be
good?
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
<http://review.cloudera.org/r/1298/#comment6533>
whitespace
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
<http://review.cloudera.org/r/1298/#comment6532>
maybe this should be warn. i think i'd want to see it and also logging of
stack trace (i don't see logging of it elsewhere)
- Jonathan
> If .META. offline between OPENING and OPENED, then wrong server location in
> .META. is possible
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-3362
> URL: https://issues.apache.org/jira/browse/HBASE-3362
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 0.90.0
>
>
> This is a good one. It happened to me testing OOME in split logging.
> * Balancer moves region to new location, regionservrer X.
> * New location regionserver X successfully opens the region and then goes to
> update .META.
> * At this point, the server carrying .META. crashes.
> * Regionserver X is stuck waiting on .META. to come back online. It takes so
> long master times out the region-in-transition
> * Master assigns the region elsewhere to regionserver Y
> * It opens successfully on regionserver Y and then it also parks waiting on
> .META. coming online
> * .META. comes online
> * The two servers X and Y race to update .META.
> I saw case where server X edit went in after server Ys edit which means that
> lookups in .META. get the wrong server. HBCK can detect this situation.
> RegionServer X when it wakes up coreeclty notices that its lost control of
> the region but the damage is done -- where damage is .META. edit.
> Chatting with Jon, he suggested that regionserver X should 'rollback' the
> .META. edit -- do explicit delete of what it added. This would work I think
> but chatting more, I'll make a fix that keeps updating the zookeeper OPENING
> state while edit goes on in a separate thread. Our continuous setting of
> OPENING will make it so region-in-transition does not timeout.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
