The assignment framework and procedure v2 framework have both been improved a lot on newer 2.x branches, so first I suggest you upgrade your cluster to 2.3.x or 2.4.x, they are more stable than previous 2.x release lines.
And on fixing the assignment problem, please try the hbck2 tool here. https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2 First please bypass the pending procedure, and then, check whether the region is already online on the region server, i.e, rzv-db10-hd,16020,1639998430080, if so, just update the location in meta, otherwise, manually schedule an procedure to assign it. Thanks. Hamado Dene <hamadod...@yahoo.com.invalid> 于2021年12月23日周四 19:38写道： > > Hi hbase comunity. > In our production environment we have hbase 2.2.6 with hadoop 2.8.5.We > found that when we do a simple reboot of an hbase regionserverafter a > configuration change or maintenance, very often regions go into transition. > the problem is that some regions often get stuck during the transition and > never recover on their own. > On the master's logs we usually see:2021-12-20 12:21:17,004 WARN > [ProcExecTimeout] assignment.AssignmentManager: STUCK Region-In-Transition > rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_7034_hnotifevents, > region=3739acab527c013a04249e5bf83756392021-12-20 12:21:17,004 WARN > [ProcExecTimeout] assignment.AssignmentManager: STUCK Region-In-Transition > rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_618_hnotifevents, > region=9d5f39b2022d42127366a6d98f4dc67b2021-12-20 12:21:17,004 WARN > [ProcExecTimeout] assignment.AssignmentManager: STUCK Region-In-Transition > rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_7854_huserlog, region=52be50fb036cde773fc5a13bd2d8547d2021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_2469_huserlog, region=b6e38248cc3336349612e50e790b9cc52021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10,16020,1639998430080, > table=mn1_4871_hinvio, region=c8570afd26643e03879708df2444ea6d2021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_3856_huserlog, region=f6a21f35f1265f9bed95cb99c8f008602021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_4871_hinvio, region=2a91d2da06b3b0082d42825a140fc4982021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_618_hwfsessionh, > region=950e0cbd96b7f1691eda531ddd446dd82021-12-20 12:21:17,005 WARN > [ProcExecTimeout] assignment.AssignmentManager: STUCK Region-In-Transition > rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_3856_hevents, region=2dec9ce47bccb2daa92da120e56d9af42021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_4639_hevents, region=65bdc201b104059d4d55f9ebe955e7bc2021-12-20 > 12:21:17,005 WARN [ProcExecTimeout] assignment.AssignmentManager: STUCK > Region-In-Transition rit=OPENING, location=rzv-db10-hd,16020,1639998430080, > table=mn1_942_huserlog, region=cb3f422f56ccc29d30f3506ea7041425 > We always fixed it by restarting the location in STUCK. > - Is there the possibility of avoiding the transition of regions when > doing a simple reboot of a node for configuration change?- Is it normal for > transitions to time out and not recover on their own? Is there any way to > avoid this problem? > Thank you, > Hamado Dene
