You can share HBase and MR if you run MR jobs only to process data off HBase and do not use HBase for real-time queries It is not generally advisable to share live (real-time) HBase cluster and run MR jobs at the same time as since HDFS can get easily saturated by MR jobs and you will have much worse HBase query latency and overall query throughput.
Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: [email protected] ________________________________________ From: [email protected] [[email protected]] On Behalf Of Stack [[email protected]] Sent: Tuesday, June 05, 2012 9:07 PM To: [email protected] Cc: [email protected] Subject: Re: Shared HDFS for HBase and MapReduce On Tue, Jun 5, 2012 at 8:29 PM, Atif Khan <[email protected]> wrote: > My first thoughts were to create a single HDFS cluster, and then point the > MapReduce and HBase servers to use the common HDFS installation. However, > Cloudera's Dos and Don'ts page > (http://www.cloudera.com/blog/2011/04/hbase-dos-and-donts/) insists that > MapReduce and HBase should not share an HDFS cluster. Rather they should > have their own individual clusters. I don't understand this recommendation, > as it would result in moving data around from one HDFS cluster to another > when running MapReduce over HBase. > It starts out "Be careful when running mixed workloads on an HBase cluster." Does your use case fit the case described: "...SLAs on hbase access" and at the same time running heavy mapreduce jobs on same cluster? If so, you may want to do the suggested two clusters. I'd suggest you start w/ all on the one cluster and see how you do. That post is > a year old. HBase has gotten steadily better since. St.Ack Confidentiality Notice: The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or [email protected] and delete or destroy any copy of this message and its attachments.
