Yes, looks like reducing the batch buffer size works (still validating). But why setAutoFlush(false) is harmful here? I just want maximum write speed.
Jianshi On Tue, Mar 3, 2015 at 10:54 PM, Jean-Marc Spaggiari < [email protected]> wrote: > Let HBase manage the flushes for you. Remove edgeTable.setAutoFlush(false) > and maybe reduce your batch size. > > I don't think that increasing the memstore is the good way to go. Sound > more like a plaster on the issue than a good fix (for me). > > JM > > 2015-03-03 9:43 GMT-05:00 Ted Yu <[email protected]>: > > > Default value for hbase.regionserver.global.memstore.size is 0.4 > > > > Meaning Maximum size of all memstores in the region server before new > > updates > > are blocked and flushes are forced is 7352m which is lower than 774m. > > > > You can increase the value for hbase.regionserver.global.memstore.size > > > > Please also see if you can distribute the writes to the underlying region > > so that the region's use of memstore comes down. > > > > Cheersx > > > > On Tue, Mar 3, 2015 at 12:07 AM, Jianshi Huang <[email protected]> > > wrote: > > > > > Hi Ted, > > > > > > Only one region server is problematic. > > > > > > hbase.regionserver.global.memstore.size is not set, the problematic > > region > > > is using 774m for memstore. > > > > > > Max heap is 18380m for all region servers. > > > > > > Jianshi > > > > > > > > > On Mon, Mar 2, 2015 at 10:59 PM, Ted Yu <[email protected]> wrote: > > > > > > > What's the value for hbase.regionserver.global.memstore.size ? > > > > > > > > Did RegionTooBusyException happen to many regions or only a few > > regions ? > > > > > > > > How much heap did you give region servers ? > > > > > > > > bq. HBase version is 0.98.0.2.1.2.0-402 > > > > > > > > Yeah, this is a bit old. Please consider upgrading. > > > > > > > > Cheers > > > > > > > > On Mon, Mar 2, 2015 at 1:42 AM, Jianshi Huang < > [email protected] > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I'm constantly facing "RegionTooBusyException: Above memstore > limit" > > > > errors > > > > > in one region server when writing data to HBase. > > > > > > > > > > I checked the region server log, and I've seen a lot of warnings > > during > > > > the > > > > > data writes: > > > > > > > > > > WARN wal.fshlog couldn't find oldest seqNum for the region we're > > > about > > > > to > > > > > flush, ... > > > > > > > > > > Then HBase seem to flush the data and added it as a HStore file. > > > > > > > > > > I also get a few warnings in client.ShortCircuitCache, says "could > > not > > > > load > > > > > ... due to InvalidToken exceptions. > > > > > > > > > > Anyone can give me hint what went wrong? > > > > > > > > > > My HBase version is 0.98.0.2.1.2.0-402, I'm using HDP 2.1, but the > > > > release > > > > > is a little bit old. > > > > > > > > > > Thanks, > > > > > > > > > > -- > > > > > Jianshi Huang > > > > > > > > > > LinkedIn: jianshi > > > > > Twitter: @jshuang > > > > > Github & Blog: http://huangjs.github.com/ > > > > > > > > > > > > > > > > > > > > > -- > > > Jianshi Huang > > > > > > LinkedIn: jianshi > > > Twitter: @jshuang > > > Github & Blog: http://huangjs.github.com/ > > > > > > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
