Hi folks,
After further testing on the new server I think I might be able to
explain the results of the benchmark tests for sequential and random
writes on RAID 1+0 (or RAID 10) as opposed to RAID 5.
Apologies for long (and possibly totally inaccurate) post, and I'll
state up front that most of the figures here are a bit hand-wavey and
probably require you to stand back and squint ;-)
So -
Our initial benchmark results (averaged out over multiple tests):
ServerA (RAID5) ServerB (RAID10)
SequentialWrite: 29 seconds 42 seconds
RandomWrite: 650 seconds 82 seconds
Now it gets interesting:
ServerA (RAID5) ServerB (RAID5)
SequentialWrite: 29 seconds 31 seconds
RandomWrite: 650 seconds 160 seconds
I did far more tests than this, but these are the salient results. So,
how to explain this?
Let's ignore the RandomWrites for now as, without going into it here,
the results are exactly as I'd expect.
So what is going on with the SequentialWrites, and why is RAID5 faster
than RAID10, and ServerA still faster than ServerB?
An important fact to bear in mind here is that both systems have the
same RAID controller with an onboard cache of 192MB, and both the
controller and the system bios have optimisations which will favour
sequential writes.
Taking ServerA first - we have 8 disks in a RAID5 array, simplistically
speaking that gives us 7+1 disks (7 for data, 1 for parity). So to write
7 blocks of data sequentially we need to do 8 writes (we are assuming
the controller at this point is intelligent enough to handle the parity
calculations and simply write from the cache).
On ServerB with 6 disks in a RAID1+0 array (a striped mirror) to write 6
blocks of data we have to do 12 writes as we are writing the data and
the mirror.
Changing ServerB to a RAID5 array with 6 disks (so 5+1), now to write 5
blocks of data we only have to do 6 writes, so now it only takes 31
seconds as opposed to 42 previously.
So it seems fairly obvious from this alone that RAID1+0 is not optimised
for sequential writes (again, a very hand-wavey generalisation).
But why is ServerA still faster under RAID5?
Putting some figures against this gives us a very interesting picture...
First:
6 writes for 5 blocks gives us: 6/5 = 1.2
8 writes for 7 blocks gives us: 8/7 = 1.4
The difference between these gives us:
1.2/1.4 = 1.05
Now - look again at the results for the RAID5 sequential writes:
ServerA (RAID5) ServerB (RAID5)
SequentialWrite: 29 seconds 31 seconds
So that's:
31/29 = 1.07
That's pretty damn close to the ratio of 6 disks as opposed to 8.
Obviously there are other factors that will influence the results here,
such as CPU usage, load, etc, but I have tried to get an average of the
test results under relatively low load conditions on as close to a level
playing field as it is possible to get.
I think it's fair to say from this that the hardware is actually
performing within reasonable parameters, and RAID1+0 is definitely not
the optimal configuration for sequential writes. Also, if I were to put
two more disks in ServerB then by my reckoning it would indeed be faster
than ServerA in a RAID5 configuration.
It may well be that spelling it out like this makes it look obvious what
the problem is, but since the results I was getting seemed to be
contrary to popular wisdom, and to any documentation I have read, I had
a hard time trying to explain what I was seeing. Hand-waving aside, I
think this explanation fits the bill.
The next step now is for the application developers to tell me how their
application actually works and what configuration they think will be
optimal.
Craig
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
