Thanks Ted, looks the right change. We are using CDH and it does not have that fix. Easy enough to put in, though I tried to build hbase and ran into many issues. Is there a doc somewhere on the prereqs for building hbase (1.0.0) in particular and the mvn flags?
Thanks, Randy On 10/7/15, 1:34 PM, "Ted Yu" <[email protected]> wrote: >Have you seen the following code ? > > ThreadPoolExecutor pool = (selectNow && >s.throttleCompaction(compaction.getRequest().getSize())) > ? longCompactions : shortCompactions; > >Looks like throttleCompaction() returned false. >Please see the following method in RatioBasedCompactionPolicy : > > public boolean throttleCompaction(long compactionSize) { > > return compactionSize > comConf.getThrottlePoint(); > >FYI > >On Wed, Oct 7, 2015 at 11:07 AM, Randy Fox <[email protected]> wrote: > >> Looking at the source I don’t see how a manual major compact could get in >> the large compaction pool: >> If selectNow Is True the pool selector has to be shortCompactions. From >> the logs it sure seems that selectNow has to be true, else it would have >> logged “system” instead of >> "org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7f236eb8 >> ;” >> >> >> Sure seems like an hbase bug. >> >> Am I missing something? >> >> Thanks, >> >> Randy >> >> /** >> * @param r HRegion store belongs to >> * @param s Store to request compaction on >> * @param why Why compaction requested -- used in debug messages >> * @param priority override the default priority (NO_PRIORITY == decide) >> * @param request custom compaction request. Can be <tt>null</tt> in >> which case a simple >> * compaction will be used. >> */ >> private synchronized CompactionRequest requestCompactionInternal(final >> HRegion r, final Store s, >> final String why, int priority, CompactionRequest request, boolean >> selectNow) >> throws IOException { >> if (this.server.isStopped() >> || (r.getTableDesc() != null && >> !r.getTableDesc().isCompactionEnabled())) { >> return null; >> } >> >> CompactionContext compaction = null; >> if (selectNow) { >> compaction = selectCompaction(r, s, priority, request); >> if (compaction == null) return null; // message logged inside >> } >> >> // We assume that most compactions are small. So, put system >> compactions into small >> // pool; we will do selection there, and move to large pool if >> necessary. >> long size = selectNow ? compaction.getRequest().getSize() : 0; >> ThreadPoolExecutor pool = (!selectNow && s.throttleCompaction(size)) >> ? longCompactions : shortCompactions; >> pool.execute(new CompactionRunner(s, r, compaction, pool)); >> if (LOG.isDebugEnabled()) { >> String type = (pool == shortCompactions) ? "Small " : "Large "; >> LOG.debug(type + "Compaction requested: " + (selectNow ? >> compaction.toString() : "system") >> + (why != null && !why.isEmpty() ? "; Because: " + why : "") + >> "; " + this); >> } >> return selectNow ? compaction.getRequest() : null; >> } >> >> >> >> >> >> On 10/6/15, 9:58 PM, "Randy Fox" <[email protected]> wrote: >> >> >Sorry, I do not follow. I thought there was one thread per pool, one for >> large and one for small. Thus my major compact should have been in the >> large pool and not blocking the small pool. That is how it worked in 0.94. >> >From the logs it demoted my compact to the small pool. >> > >> >-randy >> > >> > >> > >> >On 10/6/15, 4:32 PM, "Vladimir Rodionov" <[email protected]> wrote: >> > >> >>>> If I read this correct the size is 4.8G and the throttle is 2.5G so it >> >>should have been put into the Large compaction pool. >> >> >> >>You answered your question yourself. Minor compaction in the same pool (1 >> >>thread default) will be waiting until major is finished. >> >> >> >>-Vlad >> >> >> >> >> >>On Tue, Oct 6, 2015 at 3:59 PM, Randy Fox <[email protected]> wrote: >> >> >> >>> 2015-10-06 14:50:35,644 INFO >> org.apache.hadoop.hbase.regionserver.HStore: >> >>> Starting compaction of 4 file(s) in L of PROD_audience4,\x00 >> >>> \xB6\x0B\xA7,\x186,1443751119137.26f321d7b240c85a9350a95f6c288e49. into >> >>> >> tmpdir=hdfs://woz/hbase/data/default/PROD_audience4/26f321d7b240c85a9350a95f6c288e49/.tmp, >> >>> totalSize=4.8 G >> >>> 2015-10-06 14:50:35,646 DEBUG >> >>> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small >> Compaction >> >>> requested: >> >>> >> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7f236eb8 >> ; >> >>> Because: User-triggered major compaction; compaction_queue=(0:0), >> >>> split_queue=0, merge_queue=0 >> >>> >> >>> >> >>> If I read this correct the size is 4.8G and the throttle is 2.5G so it >> >>> should have been put into the Large compaction pool. >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> On 10/6/15, 3:07 PM, "Randy Fox" <[email protected]> wrote: >> >>> >> >>> >We just did a big leap forward from 0.94 to 1.0. We do our own major >> >>> compacts manually every night. One of the first things we noticed is >> that >> >>> when a major compact runs, no minors run. The compaction queue grows >> and >> >>> when the major compact finishes the minors then run. I have not >> found any >> >>> new knobs we should be setting. Any ideas? >> >>> >Our config is: >> >>> > >> >>> ><property> >> >>> > <name>hbase.hregion.majorcompaction</name> >> >>> > <value>0</value> >> >>> > <final>true</final> >> >>> ></property> >> >>> ><property> >> >>> > <name>hbase.hstore.compactionThreshold</name> >> >>> > <value>3</value> >> >>> ></property> >> >>> > >> >>> ><property> >> >>> > <name>hbase.hstore.compaction.max</name> >> >>> > <value>9</value> >> >>> ></property> >> >>> ><property> >> >>> > <name>hbase.regionserver.thread.compaction.throttle</name> >> >>> > <value>2684354560</value> >> >>> ></property> >> >>> > >> >>> > >> >>> >Thanks in advance, >> >>> > >> >>> >Randy >> >>> >>
