how does openbsd's RAID support stack up to the other *BSDs?
On 9/10/05, Theo de Raadt [EMAIL PROTECTED] wrote:
I thought it was time to give some details about the (minimal) RAID
management stuff coming in OpenBSD 3.8. Most of this code has been
written by Marco Peereboom with some help from David Gwynne and
Michael Shalayeff. Moral support and direction from me and Bob Beck
who has a pile of these AMI setups.
Here is a demonstration. First, a piece of dmesg output, so that we can
see which device is going to be handled:
ami0 at pci1 dev 8 function 0 Symbios Logic MegaRAID rev 0x01: apic 9 int 8
(irq 10) Dell 518/64b/lhc
ami0: FW 350O, BIOS v1.09, 128MB RAM
ami0: 2 channels, 0 FC loops, 2 logical drives
scsibus2 at ami0: 40 targets
sd0 at scsibus2 targ 0 lun 0: AMI, Host drive #00, SCSI2 0/direct fixed
sd0: 349400MB, 44542 cyl, 255 head, 63 sec, 512 bytes/sec, 715571200 sec total
sd1 at scsibus2 targ 1 lun 0: AMI, Host drive #01, SCSI2 0/direct fixed
sd1: 349400MB, 44542 cyl, 255 head, 63 sec, 512 bytes/sec, 715571200 sec total
scsibus3 at ami0: 16 targets
ses0 at scsibus3 targ 6 lun 0: DELL, PV22XS, E.17 SCSI3 3/processor fixed
scsibus4 at ami0: 16 targets
ses1 at scsibus4 targ 6 lun 0: DELL, PV22XS, E.17 SCSI3 3/processor fixed
OK, this is an AMI raid controller. It has come up with 3 scsi
busses; one for the virtual RAID volumes which there are two of, and
two SCSI busses which match the real SCSI busses that are on the
controller (to expose the SES or SAFTE enclosure management
controllers, and so that we can talk pass-through to the real disks).
If we wish to probe further details, we use
# bioctl ami0
Volume Status Size Device
ami0 0 Online 366372454400 sd0 RAID5
0 Online73403465728 0:0.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
1 Online73403465728 0:2.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
2 Online73403465728 0:4.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
3 Online73403465728 0:8.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
4 Online73403465728 1:10.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
5 Online73403465728 1:12.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
ami0 1 Online 366372454400 sd1 RAID5
0 Online73403465728 0:1.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
1 Online73403465728 0:3.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
2 Online73403465728 0:5.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
3 Online73403465728 1:9.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
4 Online73403465728 1:11.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
5 Online73403465728 1:13.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
ami0 2 Unused73403465728 1:14.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
ami0 3 Hot spare 73403465728 1:15.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
Here we can see which physical drives are on the controller, and how
they are configured into volumes. Two volumes have been created, both
of which are rather large. The drives are on two scsi busses, for
instance, 1:12.0 means SCSI bus 1, scsi target 12, lun 0. With
additional options to bioctl(4), we could find out some more (mostly
irrelevant) information.
There are also two additional devices which we know about: one is
unused (ie. not registered with the AMI firmware at the moment), and
one is a Hot Spare.
Let's cause some havoc. First, I want to pick a drive that I am going
to unplug, to mimic a failure. Let's see... 1:9.0 looks good to me.
# bioctl -b 1.9 ami0
When I look at the array, one of the drives is now blinking. I made
it blink just because I prefer to pull drives out of my sd1
filesystems rather than the sd0 filesystems. And otherwise I wouldn't
be able to show off the blink support. Anyways, I pull that
particular drive.
Immediately some churning starts, and if I re-run bioctl I can see what
has happened:
# bioctl ami0
Volume Status Size Device
ami0 0 Online 366372454400 sd0 RAID5
0 Online73403465728 0:0.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
1 Online73403465728 0:2.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
2 Online73403465728 0:4.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
3 Online73403465728 0:8.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
4 Online73403465728 1:10.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
5 Online73403465728 1:12.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6
ami0 1 Degraded 366372454400 sd1 RAID5
0 Online73403465728 0:1.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
1 Online73403465728 0:3.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
2 Online73403465728 0:5.0 ses0 MAXTOR ATLAS15K2_73SCA
JNZ6
3 Rebuild 73403465728 1:15.0 ses1 MAXTOR ATLAS15K2_73SCA
JNZ6