when you turn on debug, there is some message logged if balancer does not
run. e.g. if some region in transition - in such case, RS log should also
be checked.
On Mon, Oct 6, 2014 at 2:28 PM, Tao Xiao <xiaotao.cs....@gmail.com> wrote:
> There is just one log file with "-MASTER-" in its name, the following is
> the result of the command "*ls -R -l /var/log/hbase*"
>
>
> [root@ec1 spark]# ls -R -l /var/log/hbase
>
> /var/log/hbase:
>
> total 136840
>
> drwx------ 2 hbase hbase 4096 Jul 21 23:37 audit
>
> -rw-r--r-- 1 hbase hbase 125902996 Oct 6 02:20
> hbase-cmf-hbase-CANARY-ec1.hadoop.com.log.out
>
> -rw-r--r-- 1 hbase hbase 1424163 Oct 5 09:38
> hbase-cmf-hbase-MASTER-ec1.hadoop.com.log.out
>
> -rw-r--r-- 1 hbase hbase 12774867 Oct 6 01:39
> hbase-cmf-hbase-REGIONSERVER-ec1.hadoop.com.log.out
>
> /var/log/hbase/audit:
> total 0
>
>
> So obviously there is no other log file with "-MASTER-" in its name.
>
> The following is the content of the log4j settings file "
> */etc/hbase/conf.cloudera.hbase/log4j.properties*"
>
> log4j.rootLogger=${hbase.root.logger}
> hbase.root.logger=INFO,console
> log4j.appender.console=org.apache.log4j.ConsoleAppender
> log4j.appender.console.target=System.err
> log4j.appender.console.layout=org.apache.log4j.PatternLayout
> log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
> %c{2}: %m%n
>
> 2014-10-06 10:16 GMT+08:00 Sean Busbey <bus...@cloudera.com>:
>
> > Can you put your log4j settings for master in a pastebin and send us the
> > link?
> >
> > --
> > Sean
> > On Oct 5, 2014 8:22 PM, "Tao Xiao" <xiaotao.cs....@gmail.com> wrote:
> >
> > > Hi Ted,
> > >
> > > Actually there is no more contents in the master log --- what you
> > > saw is the snippet of master log from the time I executed "*balancer*"
> to
> > > the end of the master log.
> > >
> > > I excerpted the snippet from the master log file
> > > "/var/log/hbase/hbase-cmf-hbase-MASTER-ec1.hadoop.com.log.out" in the
> > HBase
> > > master node, I suppose that is the right master log file.
> > >
> > > Well, there is no more log info appended to that master log file from
> > > yesterday till now, does it suggest something exceptional ?
> > >
> > > 2014-10-05 22:24 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
> > >
> > > > bq. Client=root//10.196.111.116 set balanceSwitch=true
> > > > The above was the only log pertaining to balancer. There was no
> region
> > in
> > > > trnasition.
> > > >
> > > > Can you give larger snippet of master log ?
> > > >
> > > > Cheers
> > > >
> > > > On Sun, Oct 5, 2014 at 2:28 AM, Tao Xiao <xiaotao.cs....@gmail.com>
> > > wrote:
> > > >
> > > > > Hi all, I checked what all you said.
> > > > >
> > > > >
> > > > > #1. Balancer
> > > > > I used the command "*balance_switch true*" in the HBase shell and
> > that
> > > > > command returned true, indicating that the load balancer was
> running.
> > > > Then
> > > > > I used the command "*balancer*" to prompt HBase to load balance its
> > > > > regions. However after more than 5 minutes, region assignment for
> the
> > > > table
> > > > > I mentioned in the first email did not change at all, see this
> screen
> > > > shot
> > > > > <http://imgbin.org/index.php?page=image&id=19618>. I checked the
> > > master
> > > > > log since the time I manaully trigger a balance operation by the
> > > command
> > > > "
> > > > > *balancer*", and here is the master log <
> > http://pastebin.com/PagJeB1f
> > > >.
> > > > > Since the load balancer is running all the time, it's weird to see
> > that
> > > > > regions of the table in concern were never balanced.
> > > > >
> > > > >
> > > > > #2. Balancer configuration
> > > > > I'm using CDH 5.1.0 and I did not find where to set balance
> > > configuration
> > > > > in Cloudera Manager. Could anybody tell me how to ?
> > > > >
> > > > >
> > > > > #3. HBASE-12139 StochasticLoadBalancer
> > > > > My HBase cluster has about ten tables, and none of these tables has
> > > over
> > > > 20
> > > > > regions. The HBase cluster consisted of 5 nodes so the issue
> > > HBASE-12139
> > > > > does not apply to my problem.
> > > > >
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > 2014-10-03 20:31 GMT+08:00 Jean-Marc Spaggiari <
> > > jean-m...@spaggiari.org
> > > > >:
> > > > >
> > > > > > It depends on the time between the 2 screen-shots. But if there
> is
> > > > more
> > > > > > than 5 minutes (default config), then you are correct and it
> should
> > > > have
> > > > > > been a minimum balanced.
> > > > > >
> > > > > > 2014-10-03 3:30 GMT-04:00 Qiang Tian <tian...@gmail.com>:
> > > > > >
> > > > > > > according to the 2 pictures, looks the balancer not run?
> > > > > > >
> > > > > > > On Fri, Oct 3, 2014 at 1:45 AM, Ted Yu <yuzhih...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > See also
> > > > > > > > HBASE-12139 StochasticLoadBalancer doesn't work on large
> > lightly
> > > > > loaded
> > > > > > > > clusters
> > > > > > > >
> > > > > > > > Cheers
> > > > > > > >
> > > > > > > > On Thu, Oct 2, 2014 at 10:35 AM, Jean-Marc Spaggiari <
> > > > > > > > jean-m...@spaggiari.org> wrote:
> > > > > > > >
> > > > > > > > > Also, What is the balancer configured on your cluster?
> Which
> > > > HBase
> > > > > > > > version?
> > > > > > > > > If you have a 200 region table in a cluster with another
> > pretty
> > > > > small
> > > > > > > 13
> > > > > > > > > region table like the one you have, chances are high for
> this
> > > 2nd
> > > > > > table
> > > > > > > > to
> > > > > > > > > not be perfectly balanced over the cluster compared to the
> > > first
> > > > > one.
> > > > > > > > >
> > > > > > > > > JM
> > > > > > > > >
> > > > > > > > > 2014-10-02 13:30 GMT-04:00 Nick Dimiduk <
> ndimi...@gmail.com
> > >:
> > > > > > > > >
> > > > > > > > > > Is the balancer running? If it's running but unable to
> act,
> > > you
> > > > > > > should
> > > > > > > > > see
> > > > > > > > > > some explanation in the Master logs.
> > > > > > > > > >
> > > > > > > > > > On Wed, Oct 1, 2014 at 8:27 PM, Tao Xiao <
> > > > > xiaotao.cs....@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > I have a HBase table containing 40 million records.
> > > Checking
> > > > > > HBase
> > > > > > > > UI,
> > > > > > > > > I
> > > > > > > > > > > could see that this table was not spread evenly across
> > the
> > > > > > cluster.
> > > > > > > > Two
> > > > > > > > > > > nodes, *ec1.hadoop.com:60020 <
> > http://ec1.hadoop.com:60020
> > > >*
> > > > > and
> > > > > > > > > > > *ec5.hadoop.com:60020
> > > > > > > > > > > <http://ec5.hadoop.com:60020>*, for example, has only
> > one
> > > > > region
> > > > > > > > each,
> > > > > > > > > > > while other nodes have more regions. Here is the screen
> > > shot
> > > > > > > > > > > <http://imgbin.org/index.php?page=image&id=19571>.
> > > > > > > > > > >
> > > > > > > > > > > I tried to split that table by clicking the button
> > "Split".
> > > > > > Later,
> > > > > > > > each
> > > > > > > > > > > region of that table was split into two regions, but
> all
> > > > > regions
> > > > > > > for
> > > > > > > > > that
> > > > > > > > > > > table as a whole were still unevenly spread across the
> > > > cluster.
> > > > > > > Here
> > > > > > > > > > > is a screen
> > > > > > > > > > > shot after I click the button "Split"
> > > > > > > > > > > <http://imgbin.org/index.php?page=image&id=19572>.
> > > > > > > > > > >
> > > > > > > > > > > So I'd like to know how to adjust the regions for a
> table
> > > so
> > > > > that
> > > > > > > all
> > > > > > > > > > > regions for that table can be evenly spread across the
> > > > cluster.
> > > > > > > > > > >
> > > > > > > > > > > Thanks
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
