Re: Cluster hard drive ratios

2011-05-06 Thread Steve Loughran
On 05/05/11 19:14, Matthew Foley wrote: a node (or rack) is going down, don't replicate == DataNode Decommissioning. This feature is available. The current usage is to add the hosts to be decommissioned to the exclusion file named in dfs.hosts.exclude, then use DFSAdmin to invoke

Re: Cluster hard drive ratios

2011-05-06 Thread Matthew Foley
Ah, so you're suggesting there should be some hysteresis in the system, delaying response for a while to large-scale events? In particular, are you suggesting that for anticipated events, like I'm taking this rack offline for 30 minutes, but it will be back with data intact,

Re: Cluster hard drive ratios

2011-05-05 Thread Steve Loughran
On 04/05/11 19:59, Matt Goeke wrote: Mike, Thanks for the response. It looks like this discussion forked on the CDH list so I have two different conversations now. Also, you're dead on that one of the presentations I was referencing was Ravi's. With your setup I agree that it would have made

Re: Cluster hard drive ratios

2011-05-05 Thread Matthew Foley
a node (or rack) is going down, don't replicate == DataNode Decommissioning. This feature is available. The current usage is to add the hosts to be decommissioned to the exclusion file named in dfs.hosts.exclude, then use DFSAdmin to invoke -refreshNodes. (Search for decommission in DFSAdmin

Re: Cluster hard drive ratios

2011-05-04 Thread M. C. Srivas
Hey Matt, we are using the same Dell boxes, and we can get 2 GB/s per node (read and write) without problems. On Wed, May 4, 2011 at 8:43 AM, Matt Goeke msg...@gmail.com wrote: I have been reviewing quite a few presentations on the web from various businesses, in addition to the ones I