Re: difference between major and minor compactions?

2013-06-22 Thread Jean-Marc Spaggiari
Hi Yun, Few links: - http://blog.cloudera.com/blog/2012/06/hbase-io-hfile-input-output/ = There is a small paragraph about compactions which explain when they are triggered. - http://hbase.apache.org/book/regions.arch.html 9.7.6.5 You are almost right. Only thing is that HBase doesn't know when

Re: difference between major and minor compactions?

2013-06-22 Thread yun peng
Thanks, JM It seems like the sole difference btwn major and minor compaction is the number of files (to be all or just a subset of storefiles). It mentioned very briefly in http://hbase.apache.org/bookhttp://hbase.apache.org/book/regions.arch.htmlthat Sometimes a minor compaction will ... promote

Re: difference between major and minor compactions?

2013-06-22 Thread Jean-Marc Spaggiari
Hi Yun, There is more differences. The minor compactions are not remove the delete flags and the deleted cells. It only merge the small files into a bigger one. Only the major compaction (in 0.94) will deal with the delete cells. There is also some more compaction mechanism coming in trunk with

Re: difference between major and minor compactions?

2013-06-22 Thread yun peng
I am more concerned with CompactionPolicy available that allows application to manipulate a bit how compaction should go... It looks like there is newest API in .97 version

Re: difference between major and minor compactions?

2013-06-22 Thread Suraj Varma
In contrast, the major compaction is invoked in offpeak time and usually can be assume to have resource exclusively. There is no resource exclusivity with major compactions. It is just more resource _intensive_ because a major compaction will rewrite all the store files to end up with a single

RE: difference between major and minor compactions?

2013-06-22 Thread Vladimir Rodionov
Engineer Carrier IQ, www.carrieriq.com e-mail: vrodio...@carrieriq.com From: Suraj Varma [svarma...@gmail.com] Sent: Saturday, June 22, 2013 11:51 AM To: user@hbase.apache.org Subject: Re: difference between major and minor compactions? In contrast