You're welcome, Norbert.
On Fri, Dec 27, 2013 at 9:44 AM, Norbert Burger <[email protected]>wrote: > Thanks Ted - this config change appears to have reduced quite a bit of the > memstore flushes. > > Norbert > > > On Fri, Dec 27, 2013 at 12:03 AM, Ted Yu <[email protected]> wrote: > > > Raising maxlogs to 64 would give you improvement. > > > > Cheers > > > > On Dec 26, 2013, at 8:35 PM, Norbert Burger <[email protected]> > > wrote: > > > > > Thanks Ted and Liang. I do see the "Too may hlogs" line in our logs. > > > > > > We're using the default values for hbase.regionserver.maxlogs = 32 and > > > hbase.regionserver.hlog.blocksize = 64MB, so > > > > > > So given a heapsize = 12GB and lowerLimit = 0.35, does it make sense > > that I > > > should raise maxlogs to be something like 64 or 72? > > > > > > Norbert > > > > > > > > > > > > On Thu, Dec 26, 2013 at 11:03 PM, Ted Yu <[email protected]> wrote: > > > > > >> Norbert: > > >> What's your setting for hbase.regionserver.maxlogs ? > > >> > > >> Cheers > > >> > > >> On Dec 26, 2013, at 7:51 PM, 谢良 <[email protected]> wrote: > > >> > > >>> Do you observe some log like "Too many hlogs" before flushing a few > > >>> dozen KB log ? Just want to double-check it's weather a forcing > flush > > >> issue > > >>> or not:) > > >>> > > >>> Thanks, > > >>> Liang > > >>> ________________________________________ > > >>> 发件人: Norbert Burger [[email protected]] > > >>> 发送时间: 2013年12月27日 5:52 > > >>> 收件人: [email protected] > > >>> 主题: one table flushes at much smaller sizes than other? > > >>> > > >>> Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches). > > >>> > > >>> Looking through the logs, I'm seeing that some tables are flushing at > > >> much > > >>> lower size than others (a few dozen KB as opposed to the expected 128 > > >> MB). > > >>> > > >>> Looking through some of great case studies like [1], [2], and [3], > it's > > >> not > > >>> clear to me why this is happening. We're using the defaults for > upper, > > >>> lower limits (0.35 and 0.40, respectively), and the RS status pages > > show > > >>> that we're only using 25% of our memstore allocation. > > >>> > > >>> Our regions/RS ratio is currently at 60-70. We do have some other > > tables > > >>> online where column families are skewed heavily sizewise -- could > this > > be > > >>> causing other tables in our cluster to flush more frequently than > > others? > > >>> > > >>> Norbert > > >>> > > >>> [1] > > >> > > http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/ > > >>> [2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/ > > >>> [3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html > > >> > > >
