Re: Thoughts on a hybrid HBase-Hadoop cluster

2011-12-14 Thread Friso van Vollenhoven
Hi Otis, Perhaps I am getting this totally wrong, but here's how I look at it. Let's say your problem as a whole needs X spindles + Y CPU cores + Z amount of RAM to make everything work out. Then, would it matter whether you divide that amount of resources (XYZ) over heterogeneous of

Re: Thoughts on a hybrid HBase-Hadoop cluster

2011-12-14 Thread Amandeep Khurana
Otis, You could co-locate RS' with TT and DN for the most part as long as you are not really serving real time requests. Just tweak your task configs and give HBase enough RAM. You get the benefit of data locality and that could improve performance. But you should definitely try out your approach

Thoughts on a hybrid HBase-Hadoop cluster

2011-12-13 Thread Otis Gospodnetic
Hi, I was wondering if I could get some feedback on the craziness (or not) of setting up a hybrid HBase-Hadoop cluster that has the following primary uses: 1) continuous writes to HBase 2) disk and CPU intensive reads from HBase by MR jobs and writes of aggregated data back to HBase by those

Re: Thoughts on a hybrid HBase-Hadoop cluster

2011-12-13 Thread Stack
On Tue, Dec 13, 2011 at 11:44 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: Are there some obvious flaws that would really cause operational of performance pains? Would such a cluster have major performance issues because of data that needs to be transferred between DNs that are on