Author: mbautin Date: Fri Feb 24 22:47:46 2012 New Revision: 1293466 URL: http://svn.apache.org/viewvc?rev=1293466&view=rev Log: [master] Bound the number of non preferred assignment for each region server
Summary: This is diff is related to task 934952. Master should assign a bounded number of non preferred assignment for each region server. https://our.intern.facebook.com/intern/tasks/?t=934952 When the master starts or failover, it will scan the dfs to compute the locality mapping. And there 3 time windows for master to assign regions. Let's set the timer starting at the time when Master successfully initialized everything, no matter this is a fresh start up time or failover time. 1) In the 1st min, the master will always assign regions to its best locality region server. If the best locality region server is not online, then the master will hold its corresponding regions. 2) In later 4 mins, the master will still try to assign regions to its best locality region server but not holding region any more. Also start to assign these hold regions to any checked-in region server. 3) Later on, assign region to any checked-in region server. No locality assignment any more. The problem today was there were lots of region serves stopped in the first time window. When the second time window came in, the master had already hold a large number unassigned regions. And assume region server A was the first RS to checked-in this time window, the master was supposed to assign all the best locality regions plus a bounded number of unassigned regions together to this region server A. Test Plan: Running all the unit and will test the failover case in titan migration002 cluster. Reviewers: kannan, gqchen, kranganathan Reviewed By: kannan CC: hbase-eng@lists Differential Revision: https://phabricator.fb.com/D412069 Task ID: 934952 Modified: hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java Modified: hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java URL: http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java?rev=1293466&r1=1293465&r2=1293466&view=diff ============================================================================== --- hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java (original) +++ hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java Fri Feb 24 22:47:46 2012 @@ -546,6 +546,7 @@ public class RegionManager { // lookup hostname of addr if needed String hostName = null; RegionState rootState = null; + int nonPreferredAssignmentCount = 0; // Handle if root is unassigned... only assign root if root is offline. synchronized (this.regionsInTransition) { rootState = regionsInTransition.get(HRegionInfo.ROOT_REGIONINFO @@ -608,14 +609,21 @@ public class RegionManager { if (hostName.startsWith(preferredHost)) { LOG.debug("Doing Preferred Region Assignment for : " + name + " to the " + hostName); + // add the region to its preferred region server. + if (s.isUnassigned()) { + regionsToAssign.add(s); + } + continue; } else if (holdRegionForBestRegionserver || quickStartRegionServerSet.contains(preferredHost)) { continue; } } } - - if (s.isUnassigned()) { + // Only assign a configured number unassigned region at one time in the + // non preferred assignment case. + if (s.isUnassigned() && + (nonPreferredAssignmentCount++) < this.maxAssignInOneGo) { regionsToAssign.add(s); } }
