Daniel Korstad wrote:
You have lots of options. This will be a lengthy response and will give just
some ideas for just some of the options...
Just a few thoughts below interspersed with your comments.
For my server, I had started out with a single drive. I later migrated to
migrate to a RAID 1 mirror (after having to deal with reinstalls after drive
failures I wised up). Since I already had an OS that I wanted to keep, my
RAID-1 setup was a bit more involved. I following this migration to get me
there;
http://wiki.clug.org.za/wiki/RAID-1_in_a_hurry_with_grub_and_mdadm
Since you are starting from scratch, it should be easier for you. Most distros will have an installer that will guide you though the process. When you get to hard drive partitioning, look for an advance option or review and modify partition layout option or something similar otherwise it might just make a guess of what you want and that would not be RAID. In this advance partition setup, you will be able to create your RAID. First you make equal size partitions on both physical drives. For example, first carve out 100M partition on each of the two physical OS drives, than make a RAID 1 md0 with each of this partitions and than make this your /boot. Do this again for other partitions you want to have RAIDed. You can do this for /boot, /var, /home, /tmp, /usr. This is can be nice to have a separations incase a user fills /home/foo with crap and this will not effect other parts of the OS, or if mail spool fills up, it will not hang the OS. Only problem it determining how big to make them during the install. At a minimum, I would do three partitions; /boot, swap, and / This means all the others (/var, /home, /tmp, /usr) are in the / partition but this way you don't have to worry about sizing them all correctly.
For the simplest setup, I would do RAID 1 for /boot (md0), swap (md1), and / (md2) (Alternatively, your could make a swap file in / and not have a swap partition, tons of options...) Do you need to RAID your swap? Well, I would RAID it or make a swap file within a RAID partition. If you don't and your system is using swap and you lose a drive that has swap information/partition on it, you might have issues depending on how important that information in the failed drive was. You systems might hang.
Note that RAID-10 generally performs better than mirroring, particularly
when more than a few drives are involved. This can have performance
implications for swap, when large i/o pushes program pages out of
memory. The other side of that coin is that "recovery CDs" don't seem to
know how to use RAID-10 swap, which might be an issue on some systems.
After you go through the install and have a bootable OS that is running on
mdadm RAID, I would test it to make sure grub was installed correctly to both
the physical drives. If grub is not installed to both drives, and you lose one
drive down the road and if that one was the one with grub, you will have a
system that will not boot even though it has a second drive with a copy of all
the files. If this were to happen, you can recover by booting with a bootable
linux CD or recover disk and manually installing grub too. For example say you
only had grub installed to hda and it failed, boot with a live linux cd and
type (assuming /dev/hdd is the surviving second drive);
grub
device (hd0) /dev/hdd
root (hd0,0)
setup (hd0)
quit
You say you are using two 500G drives for the OS. You don't necessary have to
use all the space for the OS. You can make your partitions and take the left
over space and throw it into a logical volume. This logical volume would not
be fault tolerant, but would be the sum of the left over capacity from both
drives. For example, you use 100M for /boot and 200G for / and 2G for swap.
Take the rest and make a standard ext3 partition for the remaining space on
both drives and put them in a logical volume giving over 500G to play with for
non critical crap.
Why do I use RAID6? For the extra redundancy and I have 10 drives in my arrary.
I have been an advocate for RAID 6, especially with the every increasing drive capacity and the number of drives in the array is above say six;
http://www.intel.com/technology/magazine/computing/RAID-6-0505.htm
Other configurations will perform better for writes, know your i/o
performance requirements.
http://storageadvisors.adaptec.com/2005/10/13/raid-5-pining-for-the-fjords/
"...for using RAID-6, the single biggest reason is based on the chance of drive errors during an array rebuild after just a single drive failure. Rebuilding the data on a failed drive requires that all the other data on the other drives be pristine and error free. If there is a single error in a single sector, then the data for the corresponding sector on the replacement drive cannot be reconstructed. Data is lost. In the drive industry, the measurement of how often this occurs is called the Bit Error Rate (BER). Simple calculations will show that the chance of data loss due to BER is much greater than all the other reasons combined. Also, PATA and SATA drives have historically had much greater BERs, i.e., more bit errors per drive, than SCSI and SAS drives, causing some vendors to recommend RAID-6 for SATA drives if they’re used for mission critical data."
Since you are using only four drives for your data array, the overhead for RAID6 (two drives for parity) might not be worth it.
With four drives you would be just fine with a RAID5.
However, I would make a cron for the command to run every once in awhile. Add
this to your crontab...
#check for bad blocks once a week (every Mon at 2:30am)if bad blocks are found, they are corrected from parity information
30 2 * * Mon echo check /sys/block/md0/md/sync_action
With this, you will keep hidden bad blocks to a minimum and when a drive fails, you won't be likely bitten by a hidden bad block(s) during a rebuild.
I think a comment on "check" vs. "repair" is appropriate here. At the
least "see the man page" is appropriate.
For your data array, I would make one partition of Linux raid (FD) and have one partition for the whole drive in each physical drive. Than create your raid.
mdadm --create /dev/md3 -l 5 -n 4 /dev/<your data drive1-partition> /dev/<your data drive2-partition> /dev/<your data drive3-partition> /dev/<your data drive4-partition> <---the /dev/md3 can be what you want and will depend on how many other previous raid arrays you have, so long as you use a number not currently used.
My filesystem of choice is XFS, but you get to pick your own poison:
mkfs.xfs /-f /dev/md3
Mount the device :
mount /dev/md3 /foo
I would edit your /etc/fstab to have it automounted for each startup.
Dan.
Other misc comments: mirroring your boot partition on drives which the
BIOS won't use is a waste of bytes. If you have more than, say four,
drives fail to function you probably have a system problem other than
disk. And some BIOS versions will boot a secondary drive if the primary
fails hard but not if it has a parity or other error, which can enter a
retry loop (I *must* keep trying to boot). This behavior can be seen on
at least one major server hardware from a big name vendor, it's not just
cheap desktops. The solution, ugly as it is, is to use the firmware
"RAID" on the motherboard controller for boot, and I have several
systems with low cost small PATA drives in mirror just for boot (after
which they are spun down with hdparm settings) for this reason.
Really good notes, people should hang onto them!
--
bill davidsen <[EMAIL PROTECTED]>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
