[
https://issues.apache.org/jira/browse/HBASE-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915934#comment-16915934
]
Sean Busbey commented on HBASE-22918:
-------------------------------------
yes I believe it is possible to get a stale read at t5, provided that the RS
has also not attempted to heartbeat. the heartbeat to master's period is how we
bound that time (hbase.regionserver.msginterval which defaults to 3 seconds)
Note that the client doesn't even need a persistent connection to RS0, it just
needs its local cache of region information from hbase:meta to still say things
are on RS0.
> RegionServer violates failfast fault assumption
> -----------------------------------------------
>
> Key: HBASE-22918
> URL: https://issues.apache.org/jira/browse/HBASE-22918
> Project: HBase
> Issue Type: Bug
> Reporter: ranpanfeng
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> hbase 2.1.5 is tested and veriflied seriously before it will be deployed in
> our production environment. we give NP(network partition) fault a very
> important care. so NP fault injection tests are conducted in our test
> environment. Some findings are exposed.
> I use ycsb to write data into table SYSTEM:test, which resides on
> regionserver0; during the writting, I use iptables to drop any packet from
> regionserver0 to zookeeper quorums. after
> a default zookeeper.session.out(90'), regionserver0 throws
> YouAreDeadException after retries to connect to zookeeper on
> TimeoutException error. then, regionserver0 suicides itself, before
> regionserver0 invokes completeFile on WAL, the active master already
> considered regionserver0 has dead pre-maturely, so invokes recoverLease to
> close the WAL on regionserver0 forcely.
> In trusted idc, distributed storage assumes that the error are always
> failstop/failfast faults, there are no Byzantine failures. so in above
> scenario, active master should take over the WAL on regionserver0 after
> regionserver0 is suicided successfully. According to lease protocol, RS
> should suicide in a lease period, and active master should take over the WAL
> after a grace period has elapsed, and invariant "lease period < grace
> period" should always hold. in hbase-site.xml, only on config property
> "zookeeper.session.timeout" is given, I think we should provide two
> properties:
> 1. regionserver.zookeeper.session.timeout
> 2. master.zookeeper.session.timeout
> HBase admin then can tune regionserver.zookeeper.session.timeout less than
> master.zookeeper.session.timeout. In this way, failstop assumption is
> guaranteed.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
