Thanks for the tips. I'll play around with this this week and try to get a script that won't affect our performance too bad. I imagine most people do this at off-peak times, but we don't have that so we'll have to figure out how to spread out the load as much as possible.
On Fri, Dec 13, 2013 at 6:53 PM, Vladimir Rodionov <[email protected]>wrote: > Available HBase API allows to compact table's regions independently. > It can be cron job, script or client application which connects to HBase > cluster, selects regions and trigger compaction, > but you have to write this piece of software yourself. > > Best regards, > Vladimir Rodionov > Principal Platform Engineer > Carrier IQ, www.carrieriq.com > e-mail: [email protected] > > ________________________________________ > From: Patrick Schless [[email protected]] > Sent: Friday, December 13, 2013 4:36 PM > To: user > Subject: Re: 3-Hour Periodic Network/CPU/Disk/Latency Spikes > > Ah, sorry about the attachment (didn't realize they weren't allowed). > Here's the picture I was trying to attach: > http://www.plainlystated.com/hbase_major_compactions.png > > It sounds like you're right, Vladimir, about the compaction storm, though I > don't understand why it's about every three hours instead of every day. In > the book [1] I see the suggestion that they be managed manually. I don't > see, however, any advice on what do do after turning auto-compaction off. > Are there best practices around scheduling and monitoring the process? > > Thanks, > Patrick > > [1] > > http://hbase.apache.org/book/important_configurations.html#managed.compactions > > > > Confidentiality Notice: The information contained in this message, > including any attachments hereto, may be confidential and is intended to be > read only by the individual or entity to whom this message is addressed. If > the reader of this message is not the intended recipient or an agent or > designee of the intended recipient, please note that any review, use, > disclosure or distribution of this message or its attachments, in any form, > is strictly prohibited. If you have received this message in error, please > immediately notify the sender and/or [email protected] and > delete or destroy any copy of this message and its attachments. >
