[
https://issues.apache.org/jira/browse/HBASE-13937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592928#comment-14592928
]
Duo Zhang commented on HBASE-13937:
-----------------------------------
{quote}
In theory we should not have isServerReachable() at all. It is a parallel
cluster membership mechanism to the already existing zk based one
{quote}
I'm not familiar with the code of AM and SM, but I think zk is not enough to
keep things consistency. The EPHEMERAL node on zookeeper disappeared does not
mean the server is really dead. We still need a method like isServerReachable
to decide whether the server is really dead. One example is in HDFS HA, a
fencing is needed before transforming standby namenode to active namenode.
{quote}
If we get a connection exception or smt, we cannot assume the server is dead.
{quote}
Agree(and excuse me, what is smt?), in general only the server tells us it is
dead then we can make sure the server is dead. And after googling I think
connection refused is also not that stable(a firewall or backlog queue full can
also cause connection refused). So I think we need fencing like what HDFS HA
does?
Yeah, you may challenge that if the machine is crashed, how can we make sure
the server is dead...Honestly I do not have perfect solution. Maybe we could
introduce a DeadServerManager and make several levels of consistency, the
lowest level does not do fencing at all, the medium level do fencing with a
timeout, and the highest level will do fencing for ever(let a person stop it
maybe)
Thanks.
> Partially revert HBASE-13172
> -----------------------------
>
> Key: HBASE-13937
> URL: https://issues.apache.org/jira/browse/HBASE-13937
> Project: HBase
> Issue Type: Sub-task
> Components: Region Assignment
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.98.14, 1.2.0, 1.1.1, 1.3.0
>
> Attachments: hbase-13937_v1.patch, hbase-13937_v2.patch
>
>
> HBASE-13172 is supposed to fix a UT issue, but causes other problems that
> parent jira (HBASE-13605) is attempting to fix.
> However, HBASE-13605 patch v4 uncovers at least 2 different issues which are,
> to put it mildly, major design flaws in AM / RS.
> Regardless of 13605, the issue with 13172 is that we catch
> {{ServerNotRunningYetException}} from {{isServerReachable()}} and return
> false, which then puts the Server to the {{RegionStates.deadServers}} list.
> Once it is in that list, we can still assign and unassign regions to the RS
> after it has started (because regular assignment does not check whether the
> server is in {{RegionStates.deadServers}}. However, after the first assign
> and unassign, we cannot assign the region again since then the check for the
> lastServer will think that the server is dead.
> It turns out that a proper patch for 13605 is very hard without fixing rest
> of broken AM assumptions (see HBASE-13605, HBASE-13877 and HBASE-13895 for a
> colorful history). For 1.1.1, I think we should just revert parts of
> HBASE-13172 for now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
