Thank you everyone who replied. /David
On Mon, Mar 24, 2014 at 2:41 AM, Kevin O'dell <[email protected]>wrote: > Hey David, > > What is your write pattern? If you are bulkloading the data into HBase > this gives you the ability to add more regions and control your > compactions. If not, a high number of regions as Vlad indicated can cause > some weird issues. How many region servers do you have? What is the > current region count? How many MB/s are you ingesting into your cluster? > Do you write equally to all regions during ingest? > > > On Sun, Mar 23, 2014 at 3:51 PM, Vladimir Rodionov > <[email protected]>wrote: > > > How small is small and how large is large? > > Recommended region size is usually between 5-10GB. Too small regions > > results in more frequent flushes/compactions > > and have additional overhead in RS RAM. > > > > >>I am thinking about extending TableInputFormat to override the > > >>1-map-per-region default policy as an alternative. > > > > This looks better approach. > > > > Best regards, > > Vladimir Rodionov > > Principal Platform Engineer > > Carrier IQ, www.carrieriq.com > > e-mail: [email protected] > > > > ________________________________________ > > From: David Koch [[email protected]] > > Sent: Saturday, March 22, 2014 6:58 PM > > To: [email protected] > > Subject: Effect of region size on compaction performance > > > > Hello, > > > > We run M/Rs over several HBase tables at the same time and chose to > reduce > > region sizes in order to make map tasks faster and improve map-slot > > turnaround between the concurrent jobs. However, I am worried many > regions > > will cause longer overall compactions of the HBase data. Is this the > case? > > > > I am thinking about extending TableInputFormat to override the > > 1-map-per-region default policy as an alternative. > > > > Regards, > > > > /David > > > > Confidentiality Notice: The information contained in this message, > > including any attachments hereto, may be confidential and is intended to > be > > read only by the individual or entity to whom this message is addressed. > If > > the reader of this message is not the intended recipient or an agent or > > designee of the intended recipient, please note that any review, use, > > disclosure or distribution of this message or its attachments, in any > form, > > is strictly prohibited. If you have received this message in error, > please > > immediately notify the sender and/or [email protected] and > > delete or destroy any copy of this message and its attachments. > > > > > > -- > Kevin O'Dell > Systems Engineer, Cloudera >
