bq. to add some drives (if we can) to existing nodes +1 to the above.
hdfs manages data nodes so that percentage of capacity is on par across nodes. This would give you better balance. Cheers On Fri, Aug 22, 2014 at 2:49 PM, Ted Tuttle <[email protected]> wrote: > Ted, your question is a good one. I hadn't thought about that much. > > > > I suppose if HBase were to be aware of local resources, then we'd have to > configure the RS memory in accordance w/ the relative size of the storage. > I would think we would allocate larger heaps (up to a point to avoid > GC-induced timeouts) to processes but leave the various fractions (e.g. > block cache fraction) intact. > > > > Now that you bring the memory mismatch into focus, I am thinking it is > perhaps better to add some drives (if we can) to existing nodes to get them > to parity w/ newer higher storage density nodes. > > > > Thanks for discussion! > > > > *From:* Ted Yu [mailto:[email protected]] > *Sent:* Friday, August 22, 2014 2:40 PM > *To:* Sean Busbey > *Cc:* [email protected]; Development > > *Subject:* Re: RS storage capacity mismatches > > > > Thanks for the link, Sean. > > > > Ted: > > What about available memory on the new nodes ? How much larger is it > compared to old nodes ? > > > > Cheers > > > > On Fri, Aug 22, 2014 at 2:35 PM, Sean Busbey <[email protected]> wrote: > > Hi Ted! > > > > HBase doesn't currently take local node resources into account when > balancing across region servers. There's currently some discussion on > HBASE-11780 about building this funtionality[1]. I'm sure they'd love more > input or details about the scale you'd need to work at. > > > > > > -Sean > > > > [1]: https://issues.apache.org/jira/browse/HBASE-11780 > > > > On Fri, Aug 22, 2014 at 4:07 PM, Ted Tuttle <[email protected]> wrote: > > Hello Ted- > > We are considering added new nodes to existing cluster that have higher > storage capacity than current nodes. We hope, but do not know, that HBase > would assign more regions to RSs that have more storage capacity. > > We are considering 2x storage on new nodes vs current node. > > Thanks, > Ted > > From: Ted Yu [mailto:[email protected]] > Sent: Friday, August 22, 2014 1:58 PM > To: [email protected] > Cc: Development > Subject: Re: RS storage capacity mismatches > > > Some clarification: by unequal storage capacity, you are expecting certain > region servers to host fewer regions, right ? > > How big is the difference between memory capacity among the old and new > servers ? > > Cheers > > On Fri, Aug 22, 2014 at 1:52 PM, Ted Tuttle <[email protected]<mailto: > [email protected]>> wrote: > Hello- > > How does HBase handle region servers with unequal storage capacity? We > made the original nodes of our cluster a bit light on storage and are > wondering what would happen is we started adding nodes w/ more storage > capacity. > > Thanks, > Ted > > > > > > -- > > Sean > > >
