by my understanding, even if we started two region server on one machine, the master will check if the region server with the same ip and port had registered, if registered ,the later one will be ignored. you can see the source code in org.apache.hadoop.hbase.master.ServerManager
2012/7/30 <[email protected]> > Can you trace master log to see why there were two region servers on that > ip with different start codes ? > > Thanks > > > > On Jul 29, 2012, at 10:46 PM, deanforwever2010 <[email protected]> > wrote: > > > hi Ted,I am in the same team of Howard's > > We didn't found two region server processes running on > > 192.168.18.40 > > > > > > 2012/7/27 Ted Yu <[email protected]> > > > >> bq. the region is move from regionserver 192.168.18.40 to 192.168.18.40 > >> > >> Have you checked whether there were two region server processes running > on > >> 192.168.18.40 ? > >> > >> Cheers > >> > >> On Fri, Jul 27, 2012 at 2:43 AM, Howard <[email protected]> wrote: > >> > >>> Thanks Suraj Varma,I have put the log into the pastebin.com. > >>> > >>> master log: http://pastebin.com/QWv3K9HQ > >>> regionserver log:http://pastebin.com/LM27ui72 > >>> > >>> Because there is a lot of "region is not online" in the regionserver > >> log,so > >>> I have filter this in the regionserver log. > >>> The following is the count of "Region is not online:" log,start > >> 23:16,there > >>> is a lot of access fail because the region is not online. > >>> --------------------------d70285c1a12dec9289ce9290c9349a79 > >>> 1 23:16 > >>> 103 23:36 > >>> 142 23:37 > >>> 169 23:38 > >>> 94 23:39 > >>> 120 23:40 > >>> 39 23:41 > >>> 110 23:42 > >>> 104 23:43 > >>> 114 23:44 > >>> 90 23:45 > >>> 121 23:46 > >>> 104 23:47 > >>> 74 23:48 > >>> 96 23:49 > >>> 100 23:50 > >>> 125 23:51 > >>> 59 23:52 > >>> 113 23:53 > >>> 134 23:54 > >>> 127 23:55 > >>> 131 23:56 > >>> 119 23:57 > >>> 82 23:58 > >>> 165 23:59 > >>> > >>> and the region "d70285c1a12dec9289ce9290c9349a79" is move between two > >>> regionserver again and again by balancer.Start 23:36,the region is move > >>> from regionserver 192.168.18.40 to 192.168.18.40 and fail. > >>> > >>> > >>> 2012/7/19 Suraj Varma <[email protected]> > >>> > >>>> You can use pastebin.com or similar services to cut/paste your logs. > >>>> --S > >>>> > >>>> On Tue, Jul 17, 2012 at 7:11 PM, Howard <[email protected]> wrote: > >>>>> this problem just only once,Because it happens two day before,I > >>> remember > >>>> I > >>>>> check the master-status and only always see regions is "pending open" > >>> in > >>>>> Regions in Transition,not see there was two regionservers in the > >> same > >>>>> server. > >>>>> > >>>>> "Sent CLOSE to 192.168.0.2,60020,1342017399608",what > >>>>> does "60020,1342017399608" mean?Is there some document can help to > >>> read > >>>>> the source code? > >>>>> If still need to upload the log,how to upload the log? > >>>>> sorry I am a freshman with HBase. > >>>>> > >>>>> 2012/7/17 Ted Yu <[email protected]> > >>>>> > >>>>>> Howard: > >>>>>> Before filing JIRA, can you verify with 0.94.1 RC that Lars sent out > >>>>>> yesterday ? > >>>>>> I guess you have noticed the following toward the end of log > >> snippet: > >>>>>> > >>>>>> 2012-07-16 00:17:50,774 DEBUG > >>>>>> org.apache.hadoop.hbase. > >>>>>> master.handler.OpenedRegionHandler: Handling OPENED > >>>>>> event for > >>>>>> > >>>>>> > >>>> > >>> > >> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7e956. > >>>>>> from 192.168.1.2,60020,1342017399608; deleting unassigned node > >>>>>> > >>>>>> As Ram pointed out, there might be two region server processes > >> running > >>>> on > >>>>>> 192.168.1.2 > >>>>>> > >>>>>> Please verify whether that was the case. > >>>>>> > >>>>>> Cheers > >>>>>> > >>>>>> On Tue, Jul 17, 2012 at 7:30 AM, Ramkrishna.S.Vasudevan < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> From the logs I can see that though the server's are same their > >>> start > >>>>>> code > >>>>>>> is different. > >>>>>>> Need to analyse the previous logs also. Pls file a JIRA, if > >>> possible > >>>>>>> attach > >>>>>>> the logs to that. > >>>>>>> > >>>>>>> Thanks Howard. > >>>>>>> > >>>>>>> Regards > >>>>>>> Ram > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: Howard [mailto:[email protected]] > >>>>>>>> Sent: Tuesday, July 17, 2012 7:32 PM > >>>>>>>> To: [email protected] > >>>>>>>> Subject: Re: Load balancer repeatedly close and open region in > >> the > >>>> same > >>>>>>>> regionserver. > >>>>>>>> > >>>>>>>> *hi,Ted Yu,thanks for your reply:* > >>>>>>>> the hbase and hadoop version is > >>>>>>>> HBase Version0.94.0, r1332822Hadoop Version0.20.2-cdh3u1, > >>>>>>>> rbdafb1dbffd0d5f2fbc6ee022e1c8df6500fd638 > >>>>>>>> the following is a detail log about the same region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956, > >>>>>>>> and it repeats again and again.: > >>>>>>>> 2012-07-16 00:12:49,843 INFO > >>> org.apache.hadoop.hbase.master.HMaster: > >>>>>>>> balance > >>>>>>>> > >>>> > hri=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160 > >>>>>>>> e1b7e956., > >>>>>>>> src=192.168.1.2,60020,1342017399608, > >>>>>>>> dest=192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:12:49,843 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Starting > >>>> unassignment > >>>>>>>> of > >>>>>>>> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> (offlining) > >>>>>>>> 2012-07-16 00:12:49,843 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Creating unassigned node for > >>>>>>>> 93caf5147d40f5dd4625e160e1b7e956 in a CLOSING state > >>>>>>>> 2012-07-16 00:12:49,845 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Sent CLOSE to > >>>>>>>> 192.168.1.2,60020,1342017399608 for region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> 2012-07-16 00:12:50,555 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_CLOSED, > >>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:12:50,555 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: > >>> Handling > >>>>>>>> CLOSED > >>>>>>>> event for 93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:12:50,555 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Forcing > >> OFFLINE; > >>>>>>>> > >>>> > was=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160 > >>>>>>>> e1b7e956. > >>>>>>>> state=CLOSED, ts=1342368770556, > >>>> server=192.168.1.2,60020,1342017399608 > >>>>>>>> 2012-07-16 00:12:50,555 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Creating (or updating) unassigned > >>>> node > >>>>>>>> for > >>>>>>>> 93caf5147d40f5dd4625e160e1b7e956 with OFFLINE state > >>>>>>>> 2012-07-16 00:12:50,558 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=M_ZK_REGION_OFFLINE, > >>>> server=10.75.18.34,60000,1342017369575, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:12:50,558 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Found an > >>> existing > >>>>>>>> plan > >>>>>>>> for > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> destination server is 192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:12:50,558 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Using > >>> pre-existing > >>>>>>>> plan > >>>>>>>> for region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956.; > >>>>>>>> > >>>> > plan=hri=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd462 > >>>>>>>> 5e160e1b7e956., > >>>>>>>> src=192.168.1.2,60020,1342017399608, > >>>>>>>> dest=192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:12:50,558 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Assigning > >> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> to 192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:12:50,574 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_OPENING, > >>>>>>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:12:50,635 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_OPENING, > >>>>>>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:12:50,639 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_OPENED, > >>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:12:50,639 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: > >>> Handling > >>>>>>>> OPENED > >>>>>>>> event for > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> from 192.168.1.2,60020,1342017399608; deleting unassigned node > >>>>>>>> 2012-07-16 00:12:50,640 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Deleting existing unassigned node > >>> for > >>>>>>>> 93caf5147d40f5dd4625e160e1b7e956 that is in expected state > >>>>>>>> RS_ZK_REGION_OPENED > >>>>>>>> 2012-07-16 00:12:50,641 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: The znode of > >>>> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> has been deleted. > >>>>>>>> 2012-07-16 00:12:50,641 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Successfully deleted unassigned > >>> node > >>>> for > >>>>>>>> region 93caf5147d40f5dd4625e160e1b7e956 in expected state > >>>>>>>> RS_ZK_REGION_OPENED > >>>>>>>> 2012-07-16 00:12:50,641 INFO > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: The master has > >>>> opened > >>>>>>>> the > >>>>>>>> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> that was online on 192.168.1.2,60020,1342017399608 > >>>>>>>> 2012-07-16 00:17:49,870 INFO > >>> org.apache.hadoop.hbase.master.HMaster: > >>>>>>>> balance > >>>>>>>> > >>>> > hri=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160 > >>>>>>>> e1b7e956., > >>>>>>>> src=192.168.1.2,60020,1342017399608, > >>>>>>>> dest=192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:17:49,870 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Starting > >>>> unassignment > >>>>>>>> of > >>>>>>>> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> (offlining) > >>>>>>>> 2012-07-16 00:17:49,870 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Creating unassigned node for > >>>>>>>> 93caf5147d40f5dd4625e160e1b7e956 in a CLOSING state > >>>>>>>> 2012-07-16 00:17:49,872 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Sent CLOSE to > >>>>>>>> 192.168.1.2,60020,1342017399608 for region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> 2012-07-16 00:17:50,464 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_CLOSED, > >>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:17:50,464 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: > >>> Handling > >>>>>>>> CLOSED > >>>>>>>> event for 93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:17:50,464 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Forcing > >> OFFLINE; > >>>>>>>> > >>>> > was=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160 > >>>>>>>> e1b7e956. > >>>>>>>> state=CLOSED, ts=1342369070465, > >>>> server=192.168.1.2,60020,1342017399608 > >>>>>>>> 2012-07-16 00:17:50,464 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Creating (or updating) unassigned > >>>> node > >>>>>>>> for > >>>>>>>> 93caf5147d40f5dd4625e160e1b7e956 with OFFLINE state > >>>>>>>> 2012-07-16 00:17:50,467 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=M_ZK_REGION_OFFLINE, > >>>> server=10.75.18.34,60000,1342017369575, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:17:50,467 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Found an > >>> existing > >>>>>>>> plan > >>>>>>>> for > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> destination server is 192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:17:50,467 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Using > >>> pre-existing > >>>>>>>> plan > >>>>>>>> for region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956.; > >>>>>>>> > >>>> > plan=hri=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd462 > >>>>>>>> 5e160e1b7e956., > >>>>>>>> src=192.168.1.2,60020,1342017399608, > >>>>>>>> dest=192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:17:50,467 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Assigning > >> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> to 192.168.1.2,60020,1342002082592 > >>>>>>>> 2012-07-16 00:17:50,509 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_OPENING, > >>>>>>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:17:50,761 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_OPENING, > >>>>>>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:17:50,774 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: Handling > >>>>>>>> transition=RS_ZK_REGION_OPENED, > >>>> server=192.168.1.2,60020,1342017399608, > >>>>>>>> region=93caf5147d40f5dd4625e160e1b7e956 > >>>>>>>> 2012-07-16 00:17:50,774 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: > >>> Handling > >>>>>>>> OPENED > >>>>>>>> event for > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> from 192.168.1.2,60020,1342017399608; deleting unassigned node > >>>>>>>> 2012-07-16 00:17:50,774 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Deleting existing unassigned node > >>> for > >>>>>>>> 93caf5147d40f5dd4625e160e1b7e956 that is in expected state > >>>>>>>> RS_ZK_REGION_OPENED > >>>>>>>> 2012-07-16 00:17:50,775 DEBUG > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: The znode of > >>>> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> has been deleted. > >>>>>>>> 2012-07-16 00:17:50,775 DEBUG > >>>>>>>> org.apache.hadoop.hbase.zookeeper.ZKAssign: > >>>>>>>> master:60000-0x4384d0a47f40068 Successfully deleted unassigned > >>> node > >>>> for > >>>>>>>> region 93caf5147d40f5dd4625e160e1b7e956 in expected state > >>>>>>>> RS_ZK_REGION_OPENED > >>>>>>>> 2012-07-16 00:17:50,775 INFO > >>>>>>>> org.apache.hadoop.hbase.master.AssignmentManager: The master has > >>>> opened > >>>>>>>> the > >>>>>>>> region > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>> that was online on 192.168.1.2,60020,1342017399608 > >>>>>>>> 2012-07-16 00:22:49,916 INFO > >>> org.apache.hadoop.hbase.master.HMaster: > >>>>>>>> balance > >>>>>>>> > >>>> > hri=trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160 > >>>>>>>> e1b7e956., > >>>>>>>> src=192.168.1.2,60020,1342017399608, > >>>>>>>> dest=192.168.1.2,60020,1342002082592 > >>>>>>>> .... > >>>>>>>> > >>>>>>>> 2012/7/17 Ted Yu <[email protected]> > >>>>>>>> > >>>>>>>>> Which HBase version were you using ? > >>>>>>>>> > >>>>>>>>> The log messages below were in very short time period. Can you > >>>>>>>> provide more > >>>>>>>>> logs ? > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> > >>>>>>>>> On Tue, Jul 17, 2012 at 5:09 AM, Howard <[email protected]> > >>>> wrote: > >>>>>>>>> > >>>>>>>>>> Through the master and regionserver log,I find load balancer > >>>>>>>> repeatedly > >>>>>>>>>> close and open region in the same regionserver(period in > >>>>>>>>>> hbase.balancer.period ). > >>>>>>>>>> Does this is a bug in load balancer and how can I dig into > >> or > >>>> avoid > >>>>>>>> this? > >>>>>>>>>> The log like this: > >>>>>>>>>> 2012-07-16 00:12:50,055 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_status_list,zOwMwdi,1342294447440.7608037379153932a30f4bdd6d7e > >>>>>>>> df5d. > >>>>>>>>>> 2012-07-16 00:12:50,069 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_comment_list,zO7vqv1,1342192347676.0c9036e183055ff2df93eca348a > >>>>>>>> eeada. > >>>>>>>>>> 2012-07-16 00:12:50,206 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_status_list,zOzYDL9,1342294447440.47eb162f6aecffcddfa897fdbb70 > >>>>>>>> 472b. > >>>>>>>>>> 2012-07-16 00:12:50,301 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_status_list,zOM1HZP,1342293292587.9af5de3db0bad0470d9b8fe6b9ea > >>>>>>>> df14. > >>>>>>>>>> 2012-07-16 00:12:50,428 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_status_list,zOPKLDi,1342293292587.f89dbfc1dad366c991eba3c24822 > >>>>>>>> 8637. > >>>>>>>>>> 2012-07-16 00:12:50,562 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_status_list,zO6u4o8,1342291884831.93caf5147d40f5dd4625e160e1b7 > >>>>>>>> e956. > >>>>>>>>>> 2012-07-16 00:12:50,685 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_status_list,zOIR8Tw,1342283152067.6284e9f69042266cde88a7b19071 > >>>>>>>> d701. > >>>>>>>>>> 2012-07-16 00:12:50,791 INFO > >>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Received > >>>>>>>> request to > >>>>>>>>>> open region: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>> > trackurl_comment_list,zOqzaNw,1342195266309.bd8e19c1063f4f2984f9ca63365 > >>>>>>>> c5573. > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>> > >>> > >> >
