Please use a bigger value for MEMSTORE_FLUSHSIZE attribute.
Regards
On Tue, Oct 6, 2015 at 1:55 AM, Jeesoo Shin <bsh...@gmail.com> wrote:
> Sorry, I was in the middle of other test, I finally got back to this issue.
> And thank you, with your comment I was able to control individual memstore
> size for a table.
>
> hbase(main):051:0> create 't2', 'cf', {MEMSTORE_FLUSHSIZE=>1048576}
> hbase(main):052:0> desc 't2'
> Table t2 is ENABLED
>
>
> t2, {TABLE_ATTRIBUTES => {MEMSTORE_FLUSHSIZE => '1048576'}
>
>
> COLUMN FAMILIES DESCRIPTION
>
>
> {NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE',
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE',
> BLOCKSIZE => '65536',
> IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
> 1 row(s) in 0.0180 seconds
>
>
> I set it too small (1MB),... anyway I can see it's flushing as soon as it
> reaches 1MB.
> INFO [MemStoreFlusher.1] regionserver.HRegion: Finished memstore flush of
> ~1.53 MB/1604616, currentsize=2.75 MB/2880080 for region
> t2,,1444121020375.a117935c77004424afa1a9cf47b2d9f3. in 129ms,
> sequenceid=86681, compaction requested=true
> INFO [MemStoreFlusher.2] regionserver.HRegion: Finished memstore flush of
> ~2.20 MB/2304064, currentsize=2.43 MB/2550928 for region
> t2,,1444121020375.a117935c77004424afa1a9cf47b2d9f3. in 91ms,
> sequenceid=87187, compaction requested=true
>
>
> Regards,
>
>
>
> On Fri, Oct 2, 2015 at 10:46 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > The key in htableDescriptor you should use is MEMSTORE_FLUSHSIZE
> >
> > Search for MEMSTORE_FLUSHSIZE in the ruby scripts you would see the
> usage.
> >
> > 'hbase.hregion.memstore.flush.size' is global setting.
> >
> > On Thu, Oct 1, 2015 at 10:22 PM, Jeesoo Shin <bsh...@gmail.com> wrote:
> >
> > > I didn't get the original email, so I got this by user-thread request.
> > > I'm not sure if this reply will continue the thread or not..
> > >
> > > -------------------------------
> > > hbase(main):027:0> describe 't1'
> > > Table t1 is ENABLED
> > >
> > >
> > > t1
> > >
> > >
> > > COLUMN FAMILIES DESCRIPTION
> > >
> > >
> > > {NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
> > > REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL
> =>
> > > 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE',
> BLOCKSIZE
> > =>
> > > '65536',
> > > IN_MEMORY => 'false', BLOCKCACHE => 'true', CONFIGURATION =>
> > > {'hbase.hregion.memstore.flush.size' => '1048576'}}
> > >
> > >
> > > 1 row(s) in 0.0260 seconds
> > > -------------------------------
> > >
> > >
> > > Yes, it say it has flush size set as I wanted.
> > > But if you check the region server log (with many puts),
> > > it will still flush when the size is 128mb which is global flush size.
> > >
> > > -------------------------------
> > > INFO [MemStoreFlusher.1] regionserver.HRegion: Finished memstore flush
> > of
> > > ~128.03 MB/134252872, currentsize=9.02 MB/9463120 for region
> > >
> > >
> >
> t1,rg00000000000000000000000000000,1443099086629.cf8c21a0ca25b5be9ee485722f324ab1.
> > > in 4589ms, sequenceid=6533, compaction requested=false
> > > -------------------------------
> > >
> > > So I don't think it's working. Should it be?
> > > oh... I'm using hbase 1.0.2.
> > >
> > >
> > >
> > > --- Enclosed is a copy of the request I received.
> > > >
> > > > Return-Path: <bsh...@gmail.com>
> > > > Received: (qmail 39533 invoked by uid 99); 2 Oct 2015 02:55:51 -0000
> > > > Received: from Unknown (HELO spamd1-us-west.apache.org)
> > (209.188.14.142)
> > > > by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Oct 2015
> 02:55:51
> > > > +0000
> > > > Received: from localhost (localhost [127.0.0.1])
> > > > by spamd1-us-west.apache.org (ASF Mail Server at
> > > > spamd1-us-west.apache.org) with ESMTP id A356DC14D3
> > > > for <user-thread.49...@hbase.apache.org>; Fri, 2 Oct 2015
> > > > 02:55:50 +0000 (UTC)
> > > > X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org
> > > > X-Spam-Flag: NO
> > > > X-Spam-Score: 5.223
> > > > X-Spam-Level: *****
> > > > X-Spam-Status: No, score=5.223 tagged_above=-999 required=6.31
> > > > tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
> > > > EMPTY_MESSAGE=2.344, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01,
> > > > RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled
> > > > Authentication-Results: spamd1-us-west.apache.org (amavisd-new);
> > > > dkim=pass (2048-bit key) header.d=gmail.com
> > > > Received: from mx1-us-west.apache.org ([10.40.0.8])
> > > > by localhost (spamd1-us-west.apache.org [10.40.0.7])
> > > > (amavisd-new, port 10024)
> > > > with ESMTP id Mtb4ReQRx0K8 for <
> > > user-thread.49...@hbase.apache.org
> > > > >;
> > > > Fri, 2 Oct 2015 02:55:50 +0000 (UTC)
> > > > Received: from mail-oi0-f50.google.com (mail-oi0-f50.google.com
> > > > [209.85.218.50])
> > > > by mx1-us-west.apache.org (ASF Mail Server at
> > > > mx1-us-west.apache.org) with ESMTPS id EFA38203BA
> > > > for <user-thread.49...@hbase.apache.org>; Fri, 2 Oct 2015
> > > > 02:55:49 +0000 (UTC)
> > > > Received: by oiev17 with SMTP id v17so51558229oie.1
> > > > for <user-thread.49...@hbase.apache.org>; Thu, 01 Oct 2015
> > > > 19:55:49 -0700 (PDT)
> > > > DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
> > > > d=gmail.com; s=20120113;
> > > > h=mime-version:from:date:message-id:subject:to:content-type;
> > > > bh=vVmjSrKVXcr73mukyfCV5OK/6YuqYlSjxbqK9mcRwPw=;
> > > >
> > > > b=AP5PAfQT0hka6mUAL5XZByOtaArRowRUW4UcM4+ij5pHS7FGPfp8hJZRKewm5hbbrS
> > > >
> > > > d4kx8VdHX/H5xn/pGfbWZC1vcVOZ13KqwTMIrr/uS06vqLF72lf7Y8qeJqaJ6ptEImxp
> > > >
> > > > vos21BbILNmEBOeJ7B87yW09S8qBMU1a+RHJPL6sPb4x+FMLaVE9lFsTyVgsICaQMcir
> > > >
> > > > 3Dvq0c8xVW7rwj6W8Ln4XgaKtQAYt/cR6+bg5pA+3ls+kvl8AMVFLZtyWiIMgY+x5+GY
> > > >
> > > > 2rXCtO1aw0zk3Pdtbiy9Tk38XZZ2PiLYOG21LgrnK2pOG8025zfk+Bd+HprT6gAupkqX
> > > > xNIA==
> > > > X-Received: by 10.202.184.65 with SMTP id
> > > i62mr7764881oif.79.1443754549321;
> > > > Thu, 01 Oct 2015 19:55:49 -0700 (PDT)
> > > > MIME-Version: 1.0
> > > > Received: by 10.202.102.143 with HTTP; Thu, 1 Oct 2015 19:55:29 -0700
> > > (PDT)
> > > > From: Jeesoo Shin <bsh...@gmail.com>
> > > > Date: Fri, 2 Oct 2015 11:55:29 +0900
> > > > Message-ID: <
> > > > cafcc7atvn1yzjzspvsqqevgzt4ke_zjyigxdoknv5pcorrt...@mail.gmail.com>
> > > > Subject:
> > > > To: user-thread.49...@hbase.apache.org
> > > > Content-Type: multipart/alternative;
> > > boundary=001a113cd19a0e28d60521164d88
> > > >
> > > >
> > > >
> ----------------------------------------------------------------------
> > > >
> > > >
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Jeesoo Shin <bsh...@gmail.com>
> > > > To: user@hbase.apache.org
> > > > Cc:
> > > > Date: Thu, 24 Sep 2015 22:05:21 +0900
> > > > Subject: what can I do with create table configuration ?
> > > > Hello,
> > > >
> > > > I tried to change hbase.hregion.memstore.flush.size for a table but
> it
> > > > didn't work.
> > > > (just wanted to see if I can set different memstore size for each
> > table)
> > > > create 't1', {NAME => 'cf', CONFIGURATION =>
> > > > {'hbase.hregion.memstore.flush.size' => '1048576'}}
> > > >
> > > > What can I set with CONFIGURATION?
> > > > Any document listing it?
> > > >
> > > > TIA.
> > > >
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Ted Yu <yuzhih...@gmail.com>
> > > > To: "user@hbase.apache.org" <user@hbase.apache.org>
> > > > Cc:
> > > > Date: Thu, 24 Sep 2015 09:25:36 -0700
> > > > Subject: Re: what can I do with create table configuration ?
> > > > Which release of hbase do you use ?
> > > >
> > > > I used command similar to yours and I got :
> > > >
> > > > hbase(main):005:0> describe 't3'
> > > > Table t3 is ENABLED
> > > > t3
> > > > COLUMN FAMILIES DESCRIPTION
> > > > {NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
> > > > REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE',
> > > > MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_
> > > > DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> > > > BLOCKCACHE => 'true', CONFIGURATION =>
> > > {'hbase.hregion.memstore.flush.size'
> > > > => '1048576'}}
> > > >
> > > > I use 1.2.0-SNAPSHOT
> > > >
> > > > FYI
> > > >
> > > > On Thu, Sep 24, 2015 at 6:05 AM, Jeesoo Shin <bsh...@gmail.com>
> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I tried to change hbase.hregion.memstore.flush.size for a table but
> > it
> > > > > didn't work.
> > > > > (just wanted to see if I can set different memstore size for each
> > > table)
> > > > > create 't1', {NAME => 'cf', CONFIGURATION =>
> > > > > {'hbase.hregion.memstore.flush.size' => '1048576'}}
> > > > >
> > > > > What can I set with CONFIGURATION?
> > > > > Any document listing it?
> > > > >
> > > > > TIA.
> > > > >
> > > >
> > > >
> > >
> >
>
