I believe a lot of people experience this problem when restarting their clusters. What do you guys think of a flag which we pass to start-hbase.sh which tells master to do the failover steps but instead of letting the balancer assign the rest of the regions.
In AssignmentManager.processDeadServersAndRegionsInTransition() if that flag is set we call: assignAllUserRegions() no matter what if that flag is set. That along with the retainAssignment flag should do the trick. Our clusters are quite large and I can't remember the last time we have gotten a clean shutdown. Does anyone have other ideas? rahul On Mon, Jul 20, 2015 at 3:14 PM, rahul gidwani <[email protected]> wrote: > Did your shutdown leave any state in ZK (if you are using zk for > assignment) or did some regionservers leave behind WALs? > > You can trace AssignmentManager.joinCluster() and more specifically > > AssignmentManager.processDeadServersAndRegionsInTransition(). Can you see > any condition where you got into the failover state? > > did you shutdown using ./stop-hbase.sh? > > rahul > On Fri, Jul 17, 2015 at 9:49 PM, Ted Yu <[email protected]> wrote: > >> bq. the assignment is not always preserved >> >> Can you provide more information on this scenario ? >> Master should have retained region assignment across cluster restart. >> >> If you can pastebin relevant portion of master log w.r.t. the region(s) >> whose location was not retained, that would be nice. >> >> Thanks >> >> On Thu, Jul 16, 2015 at 4:37 AM, Ricardo Manuel Pereira Vilaça < >> [email protected]> wrote: >> >> > Hi, >> > >> > We are using HBase 0.98.6 and Hadoop 2.5.0 - CDH 5.3.5. >> > We have some doubts regarding region assignment. >> > >> > We do manual region splits/and merges and also assignment of regions to >> > regionservers. >> > Is any way to ensure that the assignment remains after a full restart of >> > the cluster? How? >> > >> > We did some experiments with >> hbase.master.wait.on.regionservers.mintostart >> > set to the total number of region >> > servers but the assignment is not always preserved. >> > >> > Thanks in advance, >> > >> > Ricardo Vilaça >> > >
