Allen Wittenauer wrote:
On 8/12/08 12:07 PM, "lohit" <[EMAIL PROTECTED]> wrote:
- why RAID5?
- If running RAID 5, why is this necessary?
Not absolute necessary.
I'd be afraid of the write penalty of RAID5 vs, say, RAID10 or even just
plain RAID1.
For the record, I don't think we have any production systems except
maybe one that uses any sort of RAID methods on the name node.
I'm sure Steve will pop up at some point and explain his reasoning on
this one. ;)
sorry, I've been away.
-the main thing is that the namenode is the point of failure for the
cluster; its the one to spend the money on, to have hooked up to your
pager when it goes offline, and to nurture like it matters.
-Whatever you can do to keep those disks alive matter, and that usually
means some RAID backup of the namenode data. Though then you have to
worry about the RAID controller itself, which is not always above
failing iself, as people can see if they search for the phrase "raid
controller failure".
-Don't bother with duplication on the worker machines, because they are
expendable and storage hurts your capital and power budgets.
One story I've head of is tracking every disk's history, even if it gets
moved around, and giving it different workload depending on its expected
failure. New disks may have more capacity, but during that bedding in
period, not to be trusted, so for the first couple of months, they're
used for transient cache data, not important persistent stuff. If you're
happy, use them for important data, but later on, when their peers from
the same batch start failing, it's time to view them as unsafe and
downgrade again. [this is all anecdotal, no paper on the topic though
there are some looking at MTBF issues in datacentres]
-steve
