Author: nspiegelberg Date: Tue Oct 11 02:05:42 2011 New Revision: 1181404 URL: http://svn.apache.org/viewvc?rev=1181404&view=rev Log: multiply assigned regions: createTable race condition
Summary: Details explained in: https://issues.apache.org/jira/browse/HBASE-2881 This also causes TestAdmin to fail intermittently. Confirmed in a reproducible manner that the race condition genuinely existed by putting a sleep of 2 mins prior to setting the region unassigned in master's in-memory state. By this time, the base scanner assigns out the region, and then setting the region unassigned in master's state after the sleep causes it to get "unassigned" once again, and hence assigned out again to a new region server. The fix basically no longer sets the newly created region unassigned in the master's in-memory state. Instead, after updating META (with region marked offline) for all the newly created regions, it simply schedules the meta scanner to be run immediately. So the regions get assigned without delay as before, but without the race condition. Test Plan: TestAdmin ran successfully a couple of times. Running all unit tests now. Will run bin/hbase verify as well on a small cluster. DiffCamp Revision: 170814 Reviewed By: jgray CC: jgray, kannan, hbase@lists Tasks: #279154: HBASE-2881: TestAdmin intermittent failures: Race condition during createTable can result in region double assignment Revert Plan: OK Modified: hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java Modified: hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java URL: http://svn.apache.org/viewvc/hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java?rev=1181404&r1=1181403&r2=1181404&view=diff ============================================================================== --- hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java (original) +++ hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Tue Oct 11 02:05:42 2011 @@ -855,6 +855,8 @@ public class HMaster extends Thread impl for(HRegionInfo newRegion : newRegions) { regionManager.createRegion(newRegion, srvr, metaRegionName); } + // kick off a meta scan right away to assign the newly created regions + regionManager.metaScannerThread.triggerNow(); } public void deleteTable(final byte [] tableName) throws IOException { Modified: hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java URL: http://svn.apache.org/viewvc/hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java?rev=1181404&r1=1181403&r2=1181404&view=diff ============================================================================== --- hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java (original) +++ hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java Tue Oct 11 02:05:42 2011 @@ -755,8 +755,9 @@ public class RegionManager { region.close(); region.getLog().closeAndDelete(); - // 5. Get it assigned to a server - setUnassigned(info, true); + // After all regions are created, the caller will schedule + // the meta scanner to run immediately and assign out the + // regions. } /**
