[
https://issues.apache.org/jira/browse/HBASE-28276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
terrytlu updated HBASE-28276:
-----------------------------
Description:
In our scenario, we usually change zookeeper.zone.parent=/hbase-unsecure to
/hbase-secure when enabling kerberos authentication. after restart the hbase
cluster, we will almost certainly be stuck on master initalization.
we can see hmaster stuck in waiting for the InitMetaProcedure finish
!image-2023-12-21-18-34-33-586.png|width=1221,height=220!
In the regionserver log, we find the regionserver receive duplicate open meta
region request, and it does not respond to hmaster the second time. so the
hmaster wait the procedure in a long time.
!image-2023-12-21-18-36-36-282.png|width=1527,height=163!
We suspect that both the servercrash procedure and hmaster startup trigger the
procedure for the online meta table.
!image-2023-12-21-18-41-55-819.png|width=714,height=331!
hmaster log this, so it will send a assign meta request to regionserver
!image-2023-12-21-18-42-47-764.png|width=720,height=30!
So any suggestions to avoid this?
was:
In our scenario, we usually change zookeeper.zone.parent=/hbase-unsecure to
/hbase-secure when enabling kerberos authentication. after restart the hbase
cluster, we will almost certainly be stuck on master initalization.
we can see hmaster stuck in waiting for the InitMetaProcedure finish
!image-2023-12-21-18-34-33-586.png|width=1221,height=220!
In the regionserver log, we find the regionserver receive duplicate open meta
region request, and it does not respond to hmaster the second time. so the
hmaster wait the procedure in a long time.
!image-2023-12-21-18-36-36-282.png|width=1527,height=163!
We suspect that both the servercrash procedure and hmaster startup trigger the
procedure for the online meta table.
!image-2023-12-21-18-41-55-819.png|width=714,height=331!
hmaster log this, so it will send a assign meta request to regionserver
!image-2023-12-21-18-42-47-764.png|width=720,height=30!
> Change zookeeper.zone.parent and restart hbase cluster, hmaster will be stuck
> waiting for hbase:meta online
> ------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-28276
> URL: https://issues.apache.org/jira/browse/HBASE-28276
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.2.7
> Reporter: terrytlu
> Priority: Major
> Attachments: image-2023-12-21-18-34-33-586.png,
> image-2023-12-21-18-36-36-282.png, image-2023-12-21-18-41-55-819.png,
> image-2023-12-21-18-42-47-764.png
>
>
> In our scenario, we usually change zookeeper.zone.parent=/hbase-unsecure to
> /hbase-secure when enabling kerberos authentication. after restart the hbase
> cluster, we will almost certainly be stuck on master initalization.
> we can see hmaster stuck in waiting for the InitMetaProcedure finish
> !image-2023-12-21-18-34-33-586.png|width=1221,height=220!
> In the regionserver log, we find the regionserver receive duplicate open meta
> region request, and it does not respond to hmaster the second time. so the
> hmaster wait the procedure in a long time.
> !image-2023-12-21-18-36-36-282.png|width=1527,height=163!
> We suspect that both the servercrash procedure and hmaster startup trigger
> the procedure for the online meta table.
> !image-2023-12-21-18-41-55-819.png|width=714,height=331!
>
> hmaster log this, so it will send a assign meta request to regionserver
> !image-2023-12-21-18-42-47-764.png|width=720,height=30!
>
> So any suggestions to avoid this?
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
