2013-07-05 10:11:24,894 INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing. servers=2 regions=145 average=72.5 mostloaded=73 leastloaded=72
Balancer saw 2 servers which were balanced - in terms of number of regions. Where did CH36 go ? Cheers On Thu, Jul 4, 2013 at 7:16 PM, ch huang <[email protected]> wrote: > i see lot's of skip balance info,is balancer refuse to work?why? > > 2013-07-05 09:56:25,121 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: > DELETING region > hdfs://CH22:9000/hbaseroot/demoK/febcd37c57994604c8e218d8eb9c75c2 > 2013-07-05 09:56:25,130 INFO org.apache.hadoop.hbase.catalog.MetaEditor: > Deleted region > demoK,rows720752,1372989269181.febcd37c57994604c8e218d8eb9c75c2. from META > 2013-07-05 09:56:25,130 INFO org.apache.hadoop.hbase.master.CatalogJanitor: > Scanned 151 catalog row(s) and gc'd 8 unreferenced parent region(s) > 2013-07-05 10:01:24,894 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: > Server information: CH34,60020,1372899976453=72, > CH35,60020,1372899981567=73 > 2013-07-05 10:01:24,894 INFO org.apache.hadoop.hbase.master.LoadBalancer: > Skipping load balancing. servers=2 regions=145 average=72.5 mostloaded=73 > leastloaded=72 > 2013-07-05 10:01:24,962 DEBUG > org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 143 catalog row(s) > and gc'd 0 unreferenced parent region(s) > 2013-07-05 10:06:24,895 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: > Server information: CH34,60020,1372899976453=72, > CH35,60020,1372899981567=73 > 2013-07-05 10:06:24,895 INFO org.apache.hadoop.hbase.master.LoadBalancer: > Skipping load balancing. servers=2 regions=145 average=72.5 mostloaded=73 > leastloaded=72 > 2013-07-05 10:06:24,964 DEBUG > org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 143 catalog row(s) > and gc'd 0 unreferenced parent region(s) > 2013-07-05 10:10:49,441 DEBUG org.apache.hadoop.hbase.client.MetaScanner: > Scanning .META. starting at row=jerry_cm,,00000000000000 for max=2147483647 > rows > 2013-07-05 10:10:49,453 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > The connection to hconnection-0x3fa2814501005d has been closed. > 2013-07-05 10:11:07,211 DEBUG org.apache.hadoop.hbase.client.MetaScanner: > Scanning .META. starting at row=dmp_pageview,,00000000000000 for max=10 > rows > 2013-07-05 10:11:07,213 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > dmp_pageview,,1372758866561.364a36024f18bc6047a0bf7158b4c744. is CH34:60020 > 2013-07-05 10:11:07,214 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > The connection to hconnection-0x3fa2814501005d has been closed. > 2013-07-05 10:11:07,215 DEBUG org.apache.hadoop.hbase.client.MetaScanner: > Scanning .META. starting at row=dmp_pageview,,00000000000000 for > max=2147483647 rows > 2013-07-05 10:11:07,219 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > The connection to hconnection-0x3fa2814501005d has been closed. > 2013-07-05 10:11:13,176 DEBUG org.apache.hadoop.hbase.client.MetaScanner: > Scanning .META. starting at row=usertable,,00000000000000 for max=10 rows > 2013-07-05 10:11:13,179 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > usertable,,1372923221432.e2c013858f08c420c9a6c86b99068f75. is CH35:60020 > 2013-07-05 10:11:13,179 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user2035056063890669682,1372923221432.833e4e71534caac4e34371819e42ca20. > is CH34:60020 > 2013-07-05 10:11:13,179 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user3071366660001926980,1372923194708.da128be082d51ec5ab1233df63aabb46. > is CH34:60020 > 2013-07-05 10:11:13,180 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user4104227499385169775,1372923194708.97d6138333c72839ce22adae4940e3d5. > is CH34:60020 > 2013-07-05 10:11:13,180 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user5139370204013525946,1372923209834.e788429ec0b1e032f4b309337c590732. > is CH34:60020 > 2013-07-05 10:11:13,180 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user6178888326785369277,1372923209834.ed841132313f30d51c82ad9f8e3e71c5. > is CH34:60020 > 2013-07-05 10:11:13,180 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user7219194014980283263,1372923217190.7a4f56a2e754a707b80dc4d4689f54e7. > is CH34:60020 > 2013-07-05 10:11:13,181 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > Cached location for > > usertable,user825931901641799999,1372923217190.b6275359a308360d59931b974d096c45. > is CH34:60020 > 2013-07-05 10:11:13,182 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > The connection to hconnection-0x3fa2814501005d has been closed. > 2013-07-05 10:11:13,183 DEBUG org.apache.hadoop.hbase.client.MetaScanner: > Scanning .META. starting at row=usertable,,00000000000000 for > max=2147483647 rows > 2013-07-05 10:11:13,188 DEBUG > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: > The connection to hconnection-0x3fa2814501005d has been closed. > 2013-07-05 10:11:24,894 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: > Server information: CH34,60020,1372899976453=72, > CH35,60020,1372899981567=73 > 2013-07-05 10:11:24,894 INFO org.apache.hadoop.hbase.master.LoadBalancer: > Skipping load balancing. servers=2 regions=145 average=72.5 mostloaded=73 > leastloaded=72 > 2013-07-05 10:11:24,962 DEBUG > org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 143 catalog row(s) > and gc'd 0 unreferenced parent region(s) > > > On Thu, Jul 4, 2013 at 5:20 PM, Ted Yu <[email protected]> wrote: > > > Can you check load balancer related log lines in master log and put them > > on pastebin so that root cause can be diagnosed ? > > > > Which HBase version are you using ? > > > > Thanks > > > > On Jul 3, 2013, at 11:44 PM, ch huang <[email protected]> wrote: > > > > > the load balancer process as i see is very slow i have a big table > ,it's > > > region distrabute is node1: 5 ,node2: 22 ,node3: 5 ,after 4 hours > ,it's > > > became > > > node1: 6 ,node2: 20 ,node1: 6, change is very very slow! > > > > > > On Thu, Jul 4, 2013 at 1:54 PM, Azuryy Yu <[email protected]> wrote: > > > > > >> I supposed load balancer is enabled and there is no pre-split. > > >> > > >> LB run periodicly, (5 mininutes by default), so after insert bulk data > > into > > >> HBase, the table is unbalance but there will be balanced eventually > > after > > >> round-robin load balancer running. > > >> > > >> > > >> On Thu, Jul 4, 2013 at 1:25 PM, Ted Yu <[email protected]> wrote: > > >> > > >>> Did you presplit your table ? > > >>> > > >>> Was load balancer enabled ? > > >>> > > >>> What HBase version do you use ? > > >>> > > >>> Thanks > > >>> > > >>> On Jul 3, 2013, at 10:21 PM, ch huang <[email protected]> wrote: > > >>> > > >>>> i insert 2GB data into my hbase cluster ,but i find the region is > not > > >>>> balance,is insert operation always cause the region number different > > on > > >>>> region server? > > >>>> > > >>>> here is my 60010 output > > >>>> > > >>>> Table Regions Name Region Server Start Key End Key > > >>>> demo2,,1372914631698.f42281ec44d113df3e644d5ecdb35366. > > >>>> CH36:60030<http://ch36:60030/> > > >>>> rows23703 > > >> demo2,rows23703,1372914684582.c3373791daec95b849ba2908c517e045. > > >>>> CH35:60030 <http://ch35:60030/> rows23703 rows347161 > > >>>> demo2,rows347161,1372914714248.dd9a6c39f65ad4a879638e1bfec76016. > > >>>> CH34:60030<http://ch34:60030/> > > >>>> rows347161 rows428378 > > >>>> demo2,rows428378,1372914745946.41a4669f36bc99ef1592b2df13c0a948. > > >>>> CH36:60030<http://ch36:60030/> > > >>>> rows428378 rows512589 > > >>>> demo2,rows512589,1372914776151.a4363bdd7403633c5775931ab32a7da8. > > >>>> CH36:60030<http://ch36:60030/> > > >>>> rows512589 rows598644 > > >>>> demo2,rows598644,1372914806469.729f52018aac909e8080f00b8d698e2d. > > >>>> CH36:60030<http://ch36:60030/> > > >>>> rows598644 rows684871 > > >>>> demo2,rows684871,1372914836812.d9cb1760fe3302500034913ff813e0f8. > > >>>> CH35:60030<http://ch35:60030/> > > >>>> rows684871 rows771559 > > >>>> demo2,rows771559,1372914836812.ac16ff1404cff5c0db3d2e3b0661ffcd. > > >>>> CH36:60030<http://ch36:60030/> > > >>>> rows771559 Regions by Region Server Region Server Region Count > > >>>> http://CH35:60030/ <http://ch35:60030/> <http://ch35:60030/> 2 > > http://CH36:60030/ <http://ch36:60030/><http://ch36:60030/>5 > > >> http://CH34:60030/ <http://ch34:60030/> <http://ch34:60030/> 1 > > >> > > >
