[
https://issues.apache.org/jira/browse/HBASE-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344694#comment-17344694
]
Esteban Gutierrez commented on HBASE-23595:
-------------------------------------------
I've run into this issue a couple of times over the years and I think we should
do better. We recently experienced this failure with a half functional host
that didn't trigger the znode expiration but caused meta RS to be unresponsive
and eventually crashing all our Masters after a cascade of failovers. What I've
been thinking is that we should carry fail-fast configs to the connection to
meta, and introspect that exception from that Put and add some logic to try to
re-locate Meta by initiating the expiration of the RS serving meta.
> HMaster abort when write to meta failed
> ---------------------------------------
>
> Key: HBASE-23595
> URL: https://issues.apache.org/jira/browse/HBASE-23595
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.2.2
> Reporter: Lijin Bin
> Priority: Major
>
> RegionStateStore
> {code}
> private void updateRegionLocation(RegionInfo regionInfo, State state, Put
> put)
> throws IOException {
> try (Table table =
> master.getConnection().getTable(TableName.META_TABLE_NAME)) {
> table.put(put);
> } catch (IOException e) {
> // TODO: Revist!!!! Means that if a server is loaded, then we will
> abort our host!
> // In tests we abort the Master!
> String msg = String.format("FAILED persisting region=%s state=%s",
> regionInfo.getShortNameToLog(), state);
> LOG.error(msg, e);
> master.abort(msg, e);
> throw e;
> }
> }
> {code}
> When regionserver (carry meta) stop or crash, if the ServerCrashProcedure
> have not start process, write to meta will fail and abort master.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
