> > hdparm -tT /dev/md0 /dev/sda7 /dev/sdb6 /dev/md0 /dev/sda7 /dev/sdb6
hdparm seems to support the bonnie result where block reads are faster,
albeit a lesser extent (dd == 85% vs 75% for bonnie).
[100MB dd write, read, delete on the raid array]
18.1MB/s writes, 10.7MB/s reads even though this should is a mostly
cache operation given 128MB memory. ~60% ratio.
102400/05.79
17685.66
102400/09.82
10427.69
> And here it is on a regular filesystem:
22.9MB/s and 13.3MB/s. Also ~60% difference.
102400/04.57
22407.00
102400/07.87
13011.43
> > Please post averaged results from several bonnie runs of 3x main memory.
>
> Sure thing. Here are the results from the 384 MB runs of bonnie on the
> RAID-1 device:
[very consistent runs, including simultaneous, all in line with the
first results posted]
> This pretty clearly suggests that the hardware is capable of a lot more
> than the RAID-1 is actually doing. I'd expect that RAID-1 block writes
> would be as slow as 10 MB/s (although they seem faster, perhaps because
> of the cache). But reads should at least be in the ballpark of 20-24 MB/s,
> shouldn't they? rather than the 9 MB/s I'm getting?
Agreed. Maybe not so strange however. Here are 5400RPM IBM's in a RAID
0 array across separate controller channels:
/dev/hde3 63 387 2610562+ 83 Linux native
/dev/hdg3 63 387 2610562+ 83 Linux native
Personalities : [2 raid0] [3 raid1]
read_ahead 8 sectors
md0 : active raid0 hde3 hdg3 5221120 blocks 16k chunks
Simple small seq array reads are 1.95x single disk, pretty much as
expected.
hdparm -tT /dev/{hde3,hdg3,md0,md0}
/dev/hde3:
Timing buffer-cache reads: 64 MB in 0.46 seconds =139.13 MB/sec
Timing buffered disk reads: 32 MB in 3.35 seconds = 9.55 MB/sec
/dev/hdg3:
Timing buffer-cache reads: 64 MB in 0.47 seconds =136.17 MB/sec
Timing buffered disk reads: 32 MB in 3.35 seconds = 9.55 MB/sec
/dev/md0:
Timing buffer-cache reads: 64 MB in 0.46 seconds =139.13 MB/sec
Timing buffered disk reads: 32 MB in 1.74 seconds =18.39 MB/sec
/dev/md0:
Timing buffer-cache reads: 64 MB in 0.59 seconds =108.47 MB/sec
Timing buffered disk reads: 32 MB in 1.77 seconds =18.08 MB/sec
With read and write I get the same fast write/slow read effect even with
RAID 0. Basically I see the same write faster than read, with the delta
getting bigger the further you get from cache containment. For large
sequential reads stripe performance difference drops from almost 2.0 to
.9. So in general support of your original post, you are seeing normal
behavior rather than anomaly.
single stream, 512k:
time dd if=/dev/zero of=/raid/100MBtest bs=1k count=500k && \
time dd if=/raid/100MBtest of=/dev/null bs=1k
512000+0 records in
512000+0 records out
0.350u 16.950s 0:44.72 38.6% 0+0k 0+0io 84pf+0w
512000+0 records in
512000+0 records out
0.410u 9.250s 0:52.44 18.4% 0+0k 0+0io 128086pf+0w
512000/44.72
11449.01
512000/52.44
9763.53
9.8/11.4
.85
And multiple i/o streams. 10x50k:
# foreach i ( 0 1 2 3 4 5 6 7 8 9 )
foreach? (time dd if=/dev/zero of=/raid/100MBtest.$i bs=1k count=50k &&
\
? time dd if=/raid/100MBtest.$i of=/dev/null bs=1k && \
? rm -rf /raid/100MBtest.$i && sync) &
foreach? end
51200+0 records in
51200+0 records out
0.040u 1.630s 1:00.29 2.7% 0+0k 0+0io 87pf+0w
51200+0 records in
51200+0 records out
0.060u 1.610s 1:01.64 2.7% 0+0k 0+0io 86pf+0w
...
51200+0 records in
51200+0 records out
0.080u 1.010s 2:44.76 0.6% 0+0k 0+0io 12890pf+0w
51200+0 records in
51200+0 records out
0.030u 0.970s 2:44.54 0.6% 0+0k 0+0io 12891pf+0w
512000/62
8258.06
512000/(120+44)
3121.95
3121/8258
.37
And finally, a large read comparing RAID 0 to building blocks. Very
repeatable.
# time dd if=/dev/md0 of=/dev/null bs=1k count=500k
512000+0 records in
512000+0 records out
0.520u 14.980s 0:47.15 32.8% 0+0k 0+0io 90pf+0w
# time dd if=/dev/hde3 of=/dev/null bs=1k count=500k
512000+0 records in
512000+0 records out
0.660u 14.680s 0:52.44 29.2% 0+0k 0+0io 89pf+0w
# time dd if=/dev/hdg3 of=/dev/null bs=1k count=500k
512000+0 records in
512000+0 records out
0.380u 15.290s 0:53.07 29.5% 0+0k 0+0io 89pf+0w
# bc -q
scale=2
47/53
.88
512000/47
10893.61
512000/53
9660.37
--
