hi, Lars and Ted, thank you so much for the insight.
For some reason, I thought the 'hbase.hregion.majorcompaction' will turn off all Major compaction, except for the manual one. Hence, the research and questions. The discussion from Ted, Nicolas and Michael( http://apache-hbase.679495.n3.nabble.com/Major-Compaction-Concerns-td3642142.html) provided a much clearer picture. Thank for the comments from Lars. It was a performance concern from my QA team, which trigger this. Knowing the implication (no additional penalty)on this area, will put it at easy. Again, appreciate the quick response. You guys are great! Demai On Thu, Mar 6, 2014 at 10:54 AM, Ted Yu <[email protected]> wrote: > Demai: > You can find description of compaction-related parameters in: > > http://hbase.apache.org/book.html#compaction > > Cheers > > > On Thu, Mar 6, 2014 at 10:44 AM, lars hofhansl <[email protected]> wrote: > > > The problem is not major compactions, but compactions that compact a lot > > of files. > > > > > > HBase will always compact in the background (minor compactions). > > If a minor compaction needs to compact all files anyway it is promoted to > > a major compaction. > > There is no performance penalty to that, just that if all files are > > compacted anyway might as well get rid of the delete markers in the > process. > > > > > > So rather than avoiding major compactions, what you want to avoid are > > compactions that touch a lot of (or all) files. You'd need to tweak the > > compaction setting - which is an art in itself. You could lower the > > compaction ratio, or increase the number of store files before compations > > start, pick a different compaction strategy, etc. > > > > > > -- Lars > > > > > > > > ________________________________ > > From: Demai Ni <[email protected]> > > To: "[email protected]" <[email protected]> > > Sent: Thursday, March 6, 2014 10:28 AM > > Subject: turn off size-based Major Compaction? > > > > > > hi, folks, > > > > Is there a way to turn off Major Compaction, for sure? > > > > We are using 96.0, and set hbase.hregion.majorcompaction is 0. The > > intention was to only run major compaction manually. It turns out after > > heavy load, major compaction still occur as shown in WebUI. > > > > After a bit search, I read this discussion: > > > > > http://apache-hbase.679495.n3.nabble.com/Major-Compaction-Concerns-td3642142.html > > , > > which indicated that there are three way to trigger major compaction: > > 1) user issued, (that is manually, such in hbase shell) > > 2) timed, (that is hbase.hregion.majorcompaction) > > 3) size-based. > > > > Is there a way to prevent or configure the 3) method? I found > > hbase.hstore.compaction.max, > > but it is for minor compaction. > > > > So the question is: what configuration for the *size-based* on > > *Major*compaction? > > > > Many thanks > > > > Demai >
