Let HBase manage the flushes for you. Remove edgeTable.setAutoFlush(false) and maybe reduce your batch size.
I don't think that increasing the memstore is the good way to go. Sound more like a plaster on the issue than a good fix (for me). JM 2015-03-03 9:43 GMT-05:00 Ted Yu <[email protected]>: > Default value for hbase.regionserver.global.memstore.size is 0.4 > > Meaning Maximum size of all memstores in the region server before new > updates > are blocked and flushes are forced is 7352m which is lower than 774m. > > You can increase the value for hbase.regionserver.global.memstore.size > > Please also see if you can distribute the writes to the underlying region > so that the region's use of memstore comes down. > > Cheersx > > On Tue, Mar 3, 2015 at 12:07 AM, Jianshi Huang <[email protected]> > wrote: > > > Hi Ted, > > > > Only one region server is problematic. > > > > hbase.regionserver.global.memstore.size is not set, the problematic > region > > is using 774m for memstore. > > > > Max heap is 18380m for all region servers. > > > > Jianshi > > > > > > On Mon, Mar 2, 2015 at 10:59 PM, Ted Yu <[email protected]> wrote: > > > > > What's the value for hbase.regionserver.global.memstore.size ? > > > > > > Did RegionTooBusyException happen to many regions or only a few > regions ? > > > > > > How much heap did you give region servers ? > > > > > > bq. HBase version is 0.98.0.2.1.2.0-402 > > > > > > Yeah, this is a bit old. Please consider upgrading. > > > > > > Cheers > > > > > > On Mon, Mar 2, 2015 at 1:42 AM, Jianshi Huang <[email protected] > > > > > wrote: > > > > > > > Hi, > > > > > > > > I'm constantly facing "RegionTooBusyException: Above memstore limit" > > > errors > > > > in one region server when writing data to HBase. > > > > > > > > I checked the region server log, and I've seen a lot of warnings > during > > > the > > > > data writes: > > > > > > > > WARN wal.fshlog couldn't find oldest seqNum for the region we're > > about > > > to > > > > flush, ... > > > > > > > > Then HBase seem to flush the data and added it as a HStore file. > > > > > > > > I also get a few warnings in client.ShortCircuitCache, says "could > not > > > load > > > > ... due to InvalidToken exceptions. > > > > > > > > Anyone can give me hint what went wrong? > > > > > > > > My HBase version is 0.98.0.2.1.2.0-402, I'm using HDP 2.1, but the > > > release > > > > is a little bit old. > > > > > > > > Thanks, > > > > > > > > -- > > > > Jianshi Huang > > > > > > > > LinkedIn: jianshi > > > > Twitter: @jshuang > > > > Github & Blog: http://huangjs.github.com/ > > > > > > > > > > > > > > > -- > > Jianshi Huang > > > > LinkedIn: jianshi > > Twitter: @jshuang > > Github & Blog: http://huangjs.github.com/ > > >
