Howdy! I am installing the Linux software RAID for the first time, and
I'm having a puzzling performance problem. Any help would be appreciated.
My thanks to the authors of the software and the raidtools docs; this
all rocks! I'm very impressed with it.
Although I sent in my subscription request hours ago, vger.rutgers.edu
seems to be a bit slow today; I'd appreciate it if anybody who replies
could cc me directly for now. Also, apologies if this question is
a common one; the mailing list archives appear to be down today and
yesterday.
The trouble I'm having is that my RAID-1 read performance appears
substantially worse than just using the same disk partitions directly.
I've tried a variety of options and I can't seem to make a dent in this
problem. The program bonnie reports write performance that is about the
same, but block read performance drops from around 12000 down to around
9000 when I use RAID-1 mirroring.
What's my system like? I'm using a stock RedHat 6.0 install with the
SMP kernel (I only have one 450 MHz PII, but the board has a second
processor slot that I'll fill eventually). It has 128 MB RAM, and the
disks in question are identical quantum disks on the same NCR SCSI bus.
More details? Here are the bonnie results:
cat /tmp/bonnie-md0
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
md0 192 5933 86.4 15222 21.8 4172 11.8 5672 81.3 9014 11.2 218.4 4.6
sd0 192 6411 92.0 15072 18.5 4265 11.7 5760 80.6 12069 13.1 201.8 4.5
When I run my tests, this is what /proc/mdstat looks like:
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 sdb6[1] sda7[0] 8353664 blocks [2/2] [UU]
unused devices: <none>
Here is my /etc/raidtab:
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 4
persistent-superblock 1
device /dev/sda7
raid-disk 0
device /dev/sdb6
raid-disk 1
Here are some (hopefully) relevant lines from the logs:
Sep 14 23:42:53 venus kernel: ncr53c876-1: rev=0x37, base=0xfa204400,
io_port=0x1800, irq=21
Sep 14 23:42:53 venus kernel: ncr53c876-1: ID 7, Fast-20, Parity Checking
Sep 14 23:42:54 venus atd: atd startup succeeded
Sep 14 23:42:53 venus kernel: ncr53c876-1: on-chip RAM at 0xfa201000
Sep 14 23:42:53 venus kernel: ncr53c876-1: restart (scsi reset).
Sep 14 23:42:53 venus kernel: ncr53c876-1: Downloading SCSI SCRIPTS.
Sep 14 23:42:53 venus kernel: ncr53c876-2: rev=0x37, base=0xfa204800,
io_port=0x2000, irq=22
Sep 14 23:42:53 venus kernel: ncr53c876-2: NCR clock is 40037KHz, 40218KHz
Sep 14 23:42:53 venus kernel: ncr53c876-2: ID 7, Fast-20, Parity Checking
Sep 14 23:42:53 venus kernel: ncr53c876-2: on-chip RAM at 0xfa202000
Sep 14 23:42:53 venus kernel: ncr53c876-2: restart (scsi reset).
Sep 14 23:42:53 venus kernel: ncr53c876-2: Downloading SCSI SCRIPTS.
Sep 14 23:42:53 venus kernel: scsi0 : ncr53c8xx - revision 3.1h
Sep 14 23:42:53 venus kernel: scsi1 : ncr53c8xx - revision 3.1h
Sep 14 23:42:53 venus kernel: scsi2 : ncr53c8xx - revision 3.1h
Sep 14 23:42:53 venus kernel: scsi : 3 hosts.
Sep 14 23:42:53 venus kernel: Vendor: QUANTUM Model: QM39100TD-SW
Rev: N491
Sep 14 23:42:53 venus kernel: Type: Direct-Access
ANSI SCSI revision: 02
Sep 14 23:42:53 venus kernel: Detected scsi disk sda at scsi1, channel 0,
id 0, lun 0
Sep 14 23:42:53 venus kernel: Vendor: QUANTUM Model: QM39100TD-SW
Rev: N491
Sep 14 23:42:53 venus kernel: Type: Direct-Access
ANSI SCSI revision: 02
Sep 14 23:42:53 venus kernel: Detected scsi disk sdb at scsi1, channel 0,
id 1, lun 0
Sep 14 23:42:53 venus kernel: ncr53c876-1-<0,0>: tagged command queue
depth set to 8
Sep 14 23:42:53 venus kernel: ncr53c876-1-<1,0>: tagged command queue
depth set to 8
Sep 14 23:42:53 venus kernel: ncr53c876-1-<0,*>: FAST-20 WIDE SCSI 40.0
MB/s (50 ns, offset 16)
Sep 14 23:42:53 venus kernel: SCSI device sda: hdwr sector= 512 bytes.
Sectors= 17783250 [8683 MB] [8.7 GB]
Sep 14 23:42:53 venus kernel: sda: sda1 sda2 < sda5 sda6 sda7 >
Sep 14 23:42:53 venus kernel: ncr53c876-1-<1,*>: FAST-20 WIDE SCSI 40.0
MB/s (50 ns, offset 16)
Sep 14 23:42:53 venus kernel: SCSI device sdb: hdwr sector= 512 bytes.
Sectors= 17783250 [8683 MB] [8.7 GB]
And I use these commands to build the RAID:
mkraid -f /dev/md0
mke2fs -b 1024 -i 1536 /dev/md0
As far as I can tell, this all looks pretty good to me. I figure that I should be
getting performance that is at least as good with mirrored disks as unmirrored disks,
so this stumps me. I've also tested this with other tools (like time cat) and
performance numbers are about the same.
Any suggestions are warmly welcomed!
William
