HBASE-917 looks relevant too.
> -----Original Message-----
> From: Andrew Purtell [mailto:[email protected]]
> Sent: Tuesday, October 05, 2010 11:50 AM
> To: [email protected]
> Subject: Re: Paid OSS task for performing manual major compactions
>
> > From: Daniel Einspanjer
> >
> > Mozilla recently had to turn off
> > automatic scheduled major compactions because if they happen
> > during our peak load time, they cause an IO delay that can
> > destabilize our client applications.
> >
> > We need to put a replacement system in place as soon as
> > possible that can initiate the major compaction and ensure
> > that it successfully completed (See HBASE-2770).
> > It would be nice if the system could actually determine the
> > health (i.e. does it need to happen right now?) and
> > dynamically schedule it based on load
>
> A suggestion for whomever might want to pick up the work:
>
> First, fix HBASE-2770.
>
> Then, it's possible to trigger major compaction on a single region
> using the shell, therefore trivially programmatically via hbase
> shell/jruby. If Mozilla is publishing DFS and HBase metrics via Ganglia
> (I suspect so, and app metrics might be useful also) then getting a
> sense of current system load wouldn't be difficult: connect to gmond,
> parse the metrics XML tree. Write a script that runs major compaction
> against each region -- a few in parallel, determined by config and
> current load estimates -- and defers the next or reduces the compaction
> concurrency if load estimates across metrics of interest as reported by
> gmond are too high.
>
> - Andy
>
>
>
>
